[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:51:31 dist-test-slave-z8x9 krb5kdc[8990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:51:31 dist-test-slave-z8x9 krb5kdc[8990](info): set up 2 sockets
Aug 12 16:51:31 dist-test-slave-z8x9 krb5kdc[8990](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 16:51:33.124017 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.447s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:51:33 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:51:33.337329 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:44205
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:44205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:33.720844 9012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:33.721319 9012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:33.721500 9012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:33.728957 9012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:51:33.729079 9012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:33.729274 9012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:33.729370 9012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:51:33.729460 9012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:51:33.740434 9012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:44205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:44205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:33.742440 9012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:33.744282 9012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:33.762880 9020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:33.763511 9028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:33.763597 9025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:33.764232 9012 server_base.cc:1034] running on GCE node
I20260812 16:51:33.765436 9012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:33.767648 9012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:33.768891 9012 hybrid_clock.cc:648] HybridClock initialized: now 1786553493768805 us; error 73 us; skew 500 ppm
I20260812 16:51:33.769510 9012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:33 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553493, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:33.773404 9012 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:51:33.775902 9012 webserver.cc:460] Webserver started at http://127.8.195.190:45953/ using document root <none> and password file <none>
I20260812 16:51:33.777060 9012 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:33.777289 9012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:33.777796 9012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:33.781955 9012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "482b4709c9a04996808f5d24c745d1cc"
format_stamp: "Formatted at 2026-08-12 16:51:33 on dist-test-slave-z8x9"
server_key: "e03550e4d1e3bfa934a4a68e7fe7faae"
server_key_iv: "a75790edf3a9c7e4734d12a2dda61bec"
server_key_version: "encryptionkey@0"
I20260812 16:51:33.782763 9012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "482b4709c9a04996808f5d24c745d1cc"
format_stamp: "Formatted at 2026-08-12 16:51:33 on dist-test-slave-z8x9"
server_key: "e03550e4d1e3bfa934a4a68e7fe7faae"
server_key_iv: "a75790edf3a9c7e4734d12a2dda61bec"
server_key_version: "encryptionkey@0"
I20260812 16:51:33.791113 9012 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260812 16:51:33.796504 9045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:33.798364 9012 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 16:51:33.798650 9012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "482b4709c9a04996808f5d24c745d1cc"
format_stamp: "Formatted at 2026-08-12 16:51:33 on dist-test-slave-z8x9"
server_key: "e03550e4d1e3bfa934a4a68e7fe7faae"
server_key_iv: "a75790edf3a9c7e4734d12a2dda61bec"
server_key_version: "encryptionkey@0"
I20260812 16:51:33.798869 9012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:33.818506 9012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:33.823561 9012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:33.823988 9012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:33.846909 9012 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:44205
I20260812 16:51:33.846933 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:44205 every 8 connection(s)
I20260812 16:51:33.849275 9012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:51:33.855940 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:33.858356 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9012
I20260812 16:51:33.858821 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:51:33.860982 8974 external_mini_cluster.cc:1428] Setting key ca1f7acefbc995831e8e8ca455cdd084
I20260812 16:51:33.873260 9138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc: Bootstrap starting.
I20260812 16:51:33.878679 9138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:33.880347 9138 log.cc:824] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:33.884195 9138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc: No bootstrap required, opened a new log
Aug 12 16:51:33 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:33.893918 9138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } }
I20260812 16:51:33.894467 9138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:33.894706 9138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b4709c9a04996808f5d24c745d1cc, State: Initialized, Role: FOLLOWER
I20260812 16:51:33.895972 9138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } }
I20260812 16:51:33.896313 9138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:51:33.896591 9138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:51:33.896812 9138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:33.899281 9138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } }
I20260812 16:51:33.900203 9138 leader_election.cc:302] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 482b4709c9a04996808f5d24c745d1cc; no voters:
I20260812 16:51:33.901115 9138 leader_election.cc:288] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:51:33.901401 9146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:33.903899 9143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:33.866906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.8.195.190:44205)
0812 16:51:33.867982 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:33.867994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:33.869471 (+ 1477us) server_negotiation.cc:407] Connection header received
0812 16:51:33.878966 (+ 9495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:33.878987 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:33.879553 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:33.879955 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:33.883311 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:33.884760 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:33.886490 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:33.887219 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:33.887528 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:33.893510 (+ 5982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:33.893567 (+ 57us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:33.893598 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:33.893636 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:33.897367 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:33.898213 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:33.898220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:33.898225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:33.898430 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:33.898896 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:33.898902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:33.898906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:33.899653 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:33.900159 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:33.901094 (+ 935us) server_negotiation.cc:299] Negotiation successful
0812 16:51:33.901735 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":433,"threads_started":1}
I20260812 16:51:33.906131 9146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [term 1 LEADER]: Becoming Leader. State: Replica: 482b4709c9a04996808f5d24c745d1cc, State: Running, Role: LEADER
I20260812 16:51:33.906294 9138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:51:33.907142 9146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } }
I20260812 16:51:33.912603 9147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "482b4709c9a04996808f5d24c745d1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } } }
I20260812 16:51:33.912787 9148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482b4709c9a04996808f5d24c745d1cc. Latest consensus state: current_term: 1 leader_uuid: "482b4709c9a04996808f5d24c745d1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482b4709c9a04996808f5d24c745d1cc" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 44205 } } }
I20260812 16:51:33.913318 9147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:33.913404 9148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:33.918422 9167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:51:33.931131 9167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:51:33.946192 9167 catalog_manager.cc:1347] Generated new cluster ID: 3ab87513f26b4789bc7d11f1dbbcd540
I20260812 16:51:33.946336 9167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:51:33.969302 9167 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:51:33.971504 9167 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:51:33.988071 9167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 482b4709c9a04996808f5d24c745d1cc: Generated new TSK 0
I20260812 16:51:33.989441 9167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:51:34.049226 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:34.384100 9189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:34.384465 9189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:34.384639 9189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:34.392097 9189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:34.392269 9189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:34.392448 9189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:51:34.402560 9189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:34.404016 9189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:34.405421 9189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:34.419432 9199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:34.419536 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:34.419389 9201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:34.420676 9189 server_base.cc:1034] running on GCE node
I20260812 16:51:34.421378 9189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:34.422324 9189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:34.423517 9189 hybrid_clock.cc:648] HybridClock initialized: now 1786553494423461 us; error 55 us; skew 500 ppm
I20260812 16:51:34.423823 9189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:34 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553494, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:34.427075 9189 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:51:34.428591 9189 webserver.cc:460] Webserver started at http://127.8.195.129:42555/ using document root <none> and password file <none>
I20260812 16:51:34.429347 9189 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:34.429459 9189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:34.429716 9189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:34.432624 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "7bed24f0e3fb48dab99d748386b80208"
format_stamp: "Formatted at 2026-08-12 16:51:34 on dist-test-slave-z8x9"
server_key: "690fb6d1e3574173966ce7c9c2ddd143"
server_key_iv: "c779e7a386e2a99f69b464c914703e55"
server_key_version: "encryptionkey@0"
I20260812 16:51:34.433416 9189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "7bed24f0e3fb48dab99d748386b80208"
format_stamp: "Formatted at 2026-08-12 16:51:34 on dist-test-slave-z8x9"
server_key: "690fb6d1e3574173966ce7c9c2ddd143"
server_key_iv: "c779e7a386e2a99f69b464c914703e55"
server_key_version: "encryptionkey@0"
I20260812 16:51:34.438373 9189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:34.441843 9212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:34.443140 9189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:34.443341 9189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "7bed24f0e3fb48dab99d748386b80208"
format_stamp: "Formatted at 2026-08-12 16:51:34 on dist-test-slave-z8x9"
server_key: "690fb6d1e3574173966ce7c9c2ddd143"
server_key_iv: "c779e7a386e2a99f69b464c914703e55"
server_key_version: "encryptionkey@0"
I20260812 16:51:34.443549 9189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:34.463392 9189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:34.467690 9189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:34.467996 9189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:34.469044 9189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:34.470860 9189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:34.470964 9189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:34.471170 9189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:34.471248 9189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:34.519448 9189 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:46507
I20260812 16:51:34.519659 9452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:46507 every 8 connection(s)
I20260812 16:51:34.521250 9189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:51:34.530854 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9189
I20260812 16:51:34.531200 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:51:34.531844 8974 external_mini_cluster.cc:1428] Setting key 43259cfbc97d6b59bc46cde3e8f7fb69
Aug 12 16:51:34 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553494, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:34.546517 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:34.525060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49625 (local address 127.8.195.190:44205)
0812 16:51:34.525605 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:34.525610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:34.526788 (+ 1178us) server_negotiation.cc:407] Connection header received
0812 16:51:34.529379 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:34.529386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:34.529485 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:34.529622 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:34.531671 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:34.532711 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:34.534373 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:34.534860 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:34.535004 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:34.539076 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:34.539081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:34.539084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:34.539115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:34.543127 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:34.543896 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:34.543902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:34.543905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:34.544072 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:34.544520 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:34.544531 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:34.544533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:34.544860 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:34.545017 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:34.545850 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 16:51:34.546072 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":326,"threads_started":1}
I20260812 16:51:34.547670 9459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:34.525634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:44205 (local address 127.8.195.129:49625)
0812 16:51:34.526613 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:34.526658 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:34.528765 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:34.529920 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:34.529930 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:34.530666 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:34.531492 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:34.531513 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:34.532886 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:34.532891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:34.534219 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:34.534232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:34.535107 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:34.535116 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:34.535364 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:34.536097 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:34.536125 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:34.538822 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:34.543288 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:34.543297 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:34.543314 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:34.543768 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:34.544186 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:34.544193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:34.544199 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:34.544393 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:34.544992 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:34.545000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:34.545397 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 16:51:34.545773 (+ 376us) client_negotiation.cc:240] Negotiation successful
0812 16:51:34.546397 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":519,"threads_started":1}
I20260812 16:51:34.549594 9453 heartbeater.cc:347] Connected to a master server at 127.8.195.190:44205
I20260812 16:51:34.550040 9453 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:34.550859 9453 heartbeater.cc:511] Master 127.8.195.190:44205 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
I20260812 16:51:34.554742 9073 ts_manager.cc:194] Registered new tserver with Master: 7bed24f0e3fb48dab99d748386b80208 (127.8.195.129:46507)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
I20260812 16:51:34.557108 9073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:49625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:51:34.581380 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:34.988425 9467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:34.988801 9467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:34.989018 9467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:34.996618 9467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:34.996752 9467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:34.996964 9467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:51:35.008299 9467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:35.009826 9467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:35.011235 9467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:35.026537 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:35.026656 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:35.026947 9482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:35.028017 9467 server_base.cc:1034] running on GCE node
I20260812 16:51:35.028811 9467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:35.029958 9467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:35.031204 9467 hybrid_clock.cc:648] HybridClock initialized: now 1786553495031121 us; error 46 us; skew 500 ppm
I20260812 16:51:35.031569 9467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:35.035149 9467 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:51:35.036882 9467 webserver.cc:460] Webserver started at http://127.8.195.130:40535/ using document root <none> and password file <none>
I20260812 16:51:35.037744 9467 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:35.037892 9467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:35.038209 9467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:35.041455 9467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "b301eda468564274b0c6c9f2b10bfebf"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "29779cd593117a7bdb0c0eb7d9c042a6"
server_key_iv: "88ffd6d9427ad1865b5c53b938fe9b93"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.042223 9467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "b301eda468564274b0c6c9f2b10bfebf"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "29779cd593117a7bdb0c0eb7d9c042a6"
server_key_iv: "88ffd6d9427ad1865b5c53b938fe9b93"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.047935 9467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 16:51:35.051702 9495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.053115 9467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 16:51:35.053471 9467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b301eda468564274b0c6c9f2b10bfebf"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "29779cd593117a7bdb0c0eb7d9c042a6"
server_key_iv: "88ffd6d9427ad1865b5c53b938fe9b93"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.053687 9467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:35.074340 9467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:35.079277 9467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:35.079645 9467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:35.080773 9467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:35.082257 9467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:35.082345 9467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.082551 9467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:35.082686 9467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.135322 9467 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:34617
I20260812 16:51:35.135502 9736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:34617 every 8 connection(s)
I20260812 16:51:35.136648 9467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:51:35.141604 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9467
I20260812 16:51:35.141878 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:51:35.142527 8974 external_mini_cluster.cc:1428] Setting key 035db6ffb93b5051f126249df3ea688c
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:35.161814 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.139660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:36971 (local address 127.8.195.190:44205)
0812 16:51:35.141304 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:35.141309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:35.141957 (+ 648us) server_negotiation.cc:407] Connection header received
0812 16:51:35.144167 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:35.144172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:35.144256 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:35.144381 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:35.145964 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.147143 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.148839 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.149409 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.149547 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.153491 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:35.153500 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:35.153508 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:35.153554 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:35.158053 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.158911 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.158920 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.158926 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.159096 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.159556 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.159563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.159570 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.160002 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:35.160158 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:35.160955 (+ 797us) server_negotiation.cc:299] Negotiation successful
0812 16:51:35.161342 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1485,"thread_start_us":1382,"threads_started":1}
I20260812 16:51:35.162609 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.140401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:44205 (local address 127.8.195.130:36971)
0812 16:51:35.141799 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:35.141849 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:35.143904 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:35.144601 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:35.144609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:35.145239 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:35.145785 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:35.145802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.147327 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.147331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:35.148696 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:35.148709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.149645 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.149652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:35.149881 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.150587 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:35.150613 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:35.153221 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:35.158203 (+ 4982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:35.158213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:35.158232 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:35.158765 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:35.159213 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:35.159218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:35.159221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:35.159431 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:35.160133 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:35.160139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:35.160482 (+ 343us) client_negotiation.cc:769] Sending connection context
0812 16:51:35.160750 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 16:51:35.161405 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":929,"threads_started":1}
I20260812 16:51:35.164243 9738 heartbeater.cc:347] Connected to a master server at 127.8.195.190:44205
I20260812 16:51:35.164695 9738 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:35.165884 9738 heartbeater.cc:511] Master 127.8.195.190:44205 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
I20260812 16:51:35.167722 9073 ts_manager.cc:194] Registered new tserver with Master: b301eda468564274b0c6c9f2b10bfebf (127.8.195.130:34617)
I20260812 16:51:35.168571 9073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:36971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:51:35.194530 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:51:35.560727 9453 heartbeater.cc:503] Master 127.8.195.190:44205 was elected leader, sending a full tablet report...
W20260812 16:51:35.602537 9752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:35.602880 9752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:35.603123 9752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:35.611354 9752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:35.611526 9752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:35.611716 9752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:51:35.622258 9752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:44205
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:35.623600 9752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:35.625039 9752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:35.640993 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:35.641039 9768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:35.641240 9772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:35.642750 9752 server_base.cc:1034] running on GCE node
I20260812 16:51:35.643438 9752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:35.644472 9752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:35.645676 9752 hybrid_clock.cc:648] HybridClock initialized: now 1786553495645613 us; error 47 us; skew 500 ppm
I20260812 16:51:35.646008 9752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:35.649727 9752 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:51:35.651643 9752 webserver.cc:460] Webserver started at http://127.8.195.131:37819/ using document root <none> and password file <none>
I20260812 16:51:35.652415 9752 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:35.652571 9752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:35.653266 9752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:35.656483 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "9ae922c7df7e446c9a1077890b6b0d41"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "0c1f4c27c7d5ab3338c6a5b5e1d98e82"
server_key_iv: "d0f57a8889de367cc8f6672a59969cfe"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.657532 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "9ae922c7df7e446c9a1077890b6b0d41"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "0c1f4c27c7d5ab3338c6a5b5e1d98e82"
server_key_iv: "d0f57a8889de367cc8f6672a59969cfe"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.663079 9752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:51:35.667105 9782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.668452 9752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:51:35.668620 9752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "9ae922c7df7e446c9a1077890b6b0d41"
format_stamp: "Formatted at 2026-08-12 16:51:35 on dist-test-slave-z8x9"
server_key: "0c1f4c27c7d5ab3338c6a5b5e1d98e82"
server_key_iv: "d0f57a8889de367cc8f6672a59969cfe"
server_key_version: "encryptionkey@0"
I20260812 16:51:35.668861 9752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:35.692101 9752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:35.697369 9752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:35.697686 9752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:35.698849 9752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:35.700335 9752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:35.700420 9752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.700592 9752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:35.700646 9752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:35.748363 9752 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:35985
I20260812 16:51:35.748517 10026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:35985 every 8 connection(s)
I20260812 16:51:35.749933 9752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:51:35.756598 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9752
I20260812 16:51:35.756829 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:51:35.757768 8974 external_mini_cluster.cc:1428] Setting key 2635660dedff811912ec8f9fcbf3a4a8
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:35.772874 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.752820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:45263 (local address 127.8.195.190:44205)
0812 16:51:35.753357 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:35.753364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:35.754591 (+ 1227us) server_negotiation.cc:407] Connection header received
0812 16:51:35.756916 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:35.756922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:35.757032 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:35.757208 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:35.759077 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.760041 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.761974 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.762504 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.762630 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.766277 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:35.766282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:35.766286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:35.766321 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:35.769644 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.770413 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.770417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.770420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.770580 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.771032 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.771035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.771037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.771349 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:35.771476 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:35.772229 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 16:51:35.772425 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":248,"threads_started":1}
I20260812 16:51:35.774108 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.753443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:44205 (local address 127.8.195.131:45263)
0812 16:51:35.754414 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:35.754457 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:35.756582 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:35.757423 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:35.757431 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:35.758154 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:35.758877 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:35.758916 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.760218 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.760225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:35.761830 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:35.761843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.762706 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.762712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:35.762912 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.763618 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:35.763642 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:35.766043 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:35.769799 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:35.769807 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:35.769820 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:35.770295 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:35.770712 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:35.770715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:35.770718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:35.770926 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:35.771465 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:35.771471 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:35.771808 (+ 337us) client_negotiation.cc:769] Sending connection context
0812 16:51:35.772085 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 16:51:35.772494 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":473,"threads_started":1}
I20260812 16:51:35.775627 10028 heartbeater.cc:347] Connected to a master server at 127.8.195.190:44205
I20260812 16:51:35.775902 10028 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:35.776583 10028 heartbeater.cc:511] Master 127.8.195.190:44205 requested a full tablet report, sending...
I20260812 16:51:35.778061 9072 ts_manager.cc:194] Registered new tserver with Master: 9ae922c7df7e446c9a1077890b6b0d41 (127.8.195.131:35985)
I20260812 16:51:35.779071 9072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:45263
I20260812 16:51:35.787624 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:35.842813 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.825233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.8.195.129:46507)
0812 16:51:35.826135 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:35.826142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:35.826171 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:51:35.826239 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:35.826246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:35.826461 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:35.826598 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:35.828403 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.829851 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.831314 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.831921 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.832018 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.836298 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:35.836310 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:35.836323 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:35.836361 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:35.839741 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.840451 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.840459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.840461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.840656 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.841056 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.841062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.841065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.841469 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:35.841842 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:35.842127 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 16:51:35.842445 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":477,"threads_started":1}
W20260812 16:51:35.844293 9270 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50206
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:51:35 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:35.897868 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.881581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.8.195.129:46507)
0812 16:51:35.881839 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:35.881848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:35.882192 (+ 344us) server_negotiation.cc:407] Connection header received
0812 16:51:35.882414 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:35.882419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:35.882565 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:35.882690 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:35.884398 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.885144 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.886632 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.887363 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.887508 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.892333 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:35.892339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:35.892342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:35.892374 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:35.895282 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.896083 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.896087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.896089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.896225 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.896560 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.896563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.896565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.896816 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:35.896990 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:35.897439 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 16:51:35.897611 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:51:35.899904 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9189
I20260812 16:51:35.914155 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:35.901020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.8.195.129:46507)
0812 16:51:35.901305 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:35.901311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:35.901551 (+ 240us) server_negotiation.cc:407] Connection header received
0812 16:51:35.901624 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:35.901627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:35.901682 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:35.901773 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:35.903146 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.903895 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.905581 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:35.905978 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:35.906069 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:35.908564 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:35.908570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:35.908574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:35.908614 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:35.911607 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.912237 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.912242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.912245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.912413 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:35.912764 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:35.912767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:35.912769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:35.913014 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:35.913303 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:35.913614 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 16:51:35.913785 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 16:51:35.915071 9444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:36.005785 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9189
I20260812 16:51:36.029362 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9467
Aug 12 16:51:36 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:51:36.045600 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:36.030339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.8.195.130:34617)
0812 16:51:36.030978 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:36.030985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:36.031012 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:51:36.031074 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:36.031082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:36.031299 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:36.031442 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:36.032651 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:36.034117 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:36.035406 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:36.035942 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:36.036039 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:36.040125 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:36.040135 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:36.040149 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:36.040186 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:36.043067 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:36.043703 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:36.043710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:36.043713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:36.043833 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:36.044180 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:36.044183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:36.044185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:36.044513 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:36.044746 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:36.044977 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 16:51:36.045234 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":346,"threads_started":1}
I20260812 16:51:36.046881 9728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:36.129947 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9467
I20260812 16:51:36.154275 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9752
Aug 12 16:51:36 dist-test-slave-z8x9 krb5kdc[8990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:51:36.171729 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:36.155229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.8.195.131:35985)
0812 16:51:36.155888 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:36.155895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:36.155925 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:51:36.155990 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:36.155996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:36.156197 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:36.156334 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:36.157767 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:36.158791 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:36.160163 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:36.160909 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:36.161072 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:36.165649 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:36.165661 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:36.165676 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:36.165713 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:36.168710 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:36.169504 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:36.169513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:36.169517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:36.169661 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:36.170045 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:36.170049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:36.170059 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:36.170415 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:36.170741 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:36.171072 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 16:51:36.171352 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":271,"threads_started":1}
I20260812 16:51:36.173039 10020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:36.255957 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9752
I20260812 16:51:36.277849 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9012
I20260812 16:51:36.279057 9126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:36.382601 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9012
2026-08-12T16:51:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4770 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:51:36 dist-test-slave-z8x9 krb5kdc[10079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:51:36 dist-test-slave-z8x9 krb5kdc[10079](info): set up 2 sockets
Aug 12 16:51:36 dist-test-slave-z8x9 krb5kdc[10079](info): commencing operation
krb5kdc: starting...
W20260812 16:51:37.753783 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:51:37 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:51:37.973156 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39259
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:39259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:38.297533 10122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:38.297837 10122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:38.298012 10122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:38.305230 10122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:51:38.305372 10122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:38.305460 10122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:38.305559 10122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:51:38.305645 10122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:51:38.315500 10122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:39259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:38.316807 10122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:38.318176 10122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:38.330010 10130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:38.330086 10131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:38.330114 10134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:38.331125 10122 server_base.cc:1034] running on GCE node
I20260812 16:51:38.331941 10122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:38.333359 10122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:38.334564 10122 hybrid_clock.cc:648] HybridClock initialized: now 1786553498334514 us; error 39 us; skew 500 ppm
I20260812 16:51:38.334910 10122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:38 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553498, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:38.342648 10122 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:51:38.344310 10122 webserver.cc:460] Webserver started at http://127.8.195.190:41551/ using document root <none> and password file <none>
I20260812 16:51:38.345039 10122 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:38.345172 10122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:38.345543 10122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:38.348537 10122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "03d25037477146e6a61a34db075269c9"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "1bb15c4298684f928e2541c3d5b3f8ff"
server_key_iv: "fc333490439afcc1fb2d8ce87473abb2"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.349404 10122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "03d25037477146e6a61a34db075269c9"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "1bb15c4298684f928e2541c3d5b3f8ff"
server_key_iv: "fc333490439afcc1fb2d8ce87473abb2"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.354525 10122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:38.357893 10149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:38.359149 10122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:38.359357 10122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "03d25037477146e6a61a34db075269c9"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "1bb15c4298684f928e2541c3d5b3f8ff"
server_key_iv: "fc333490439afcc1fb2d8ce87473abb2"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.359557 10122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:38.380409 10122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:38.384697 10122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:38.384997 10122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:38.404502 10122 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:39259
I20260812 16:51:38.404533 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:39259 every 8 connection(s)
I20260812 16:51:38.405961 10122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:51:38.409773 10260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:38.413730 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10122
I20260812 16:51:38.413980 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:51:38.414795 8974 external_mini_cluster.cc:1428] Setting key 319b7668b24265b8a40f6be9ff99d2d5
I20260812 16:51:38.424165 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9: Bootstrap starting.
I20260812 16:51:38.428170 10260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:38.429589 10260 log.cc:824] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:51:38 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:38.433106 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9: No bootstrap required, opened a new log
I20260812 16:51:38.439697 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:38.416981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.8.195.190:39259)
0812 16:51:38.417812 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:38.417823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:38.417865 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:51:38.419484 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:38.419506 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:38.419914 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:38.420335 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:38.422281 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:38.424018 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:38.425774 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:38.426450 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:38.426579 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:38.431729 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:38.431737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:38.431752 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:38.431799 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:38.435015 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:38.435861 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:38.435867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:38.435873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:38.436036 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:38.436485 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:38.436490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:38.436492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:38.437150 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:38.437501 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:38.438052 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 16:51:38.438547 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":414,"threads_started":1}
I20260812 16:51:38.442538 10260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } }
I20260812 16:51:38.443035 10260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:38.443158 10260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d25037477146e6a61a34db075269c9, State: Initialized, Role: FOLLOWER
I20260812 16:51:38.443709 10260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } }
I20260812 16:51:38.443961 10260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:51:38.444100 10260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:51:38.444298 10260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:38.446648 10260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } }
I20260812 16:51:38.447173 10260 leader_election.cc:302] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d25037477146e6a61a34db075269c9; no voters:
I20260812 16:51:38.447764 10260 leader_election.cc:288] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:51:38.447896 10274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:38.448613 10274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [term 1 LEADER]: Becoming Leader. State: Replica: 03d25037477146e6a61a34db075269c9, State: Running, Role: LEADER
I20260812 16:51:38.449286 10274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } }
I20260812 16:51:38.449890 10260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:51:38.452265 10276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d25037477146e6a61a34db075269c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } } }
I20260812 16:51:38.452716 10276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:38.453153 10277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d25037477146e6a61a34db075269c9. Latest consensus state: current_term: 1 leader_uuid: "03d25037477146e6a61a34db075269c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d25037477146e6a61a34db075269c9" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39259 } } }
I20260812 16:51:38.454128 10277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:38.455152 10285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:51:38.460052 10285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:51:38.468444 10285 catalog_manager.cc:1347] Generated new cluster ID: f254b0858b5f4bb282cf8f272b167dba
I20260812 16:51:38.468564 10285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:51:38.479445 10285 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:51:38.481384 10285 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:51:38.491772 10285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d25037477146e6a61a34db075269c9: Generated new TSK 0
I20260812 16:51:38.492692 10285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:51:38.553061 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:38.888358 10315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:38.888636 10315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:38.888878 10315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:38.896270 10315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:38.896385 10315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:38.896597 10315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:51:38.906486 10315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:38.907760 10315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:38.909080 10315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:38.922896 10333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:38.922827 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:38.922858 10329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:38.923507 10315 server_base.cc:1034] running on GCE node
I20260812 16:51:38.924178 10315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:38.925182 10315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:38.926458 10315 hybrid_clock.cc:648] HybridClock initialized: now 1786553498926400 us; error 51 us; skew 500 ppm
I20260812 16:51:38.926751 10315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:38 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553498, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:38.929872 10315 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:51:38.931296 10315 webserver.cc:460] Webserver started at http://127.8.195.129:45373/ using document root <none> and password file <none>
I20260812 16:51:38.931947 10315 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:38.932078 10315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:38.932387 10315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:38.935937 10315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "a9a6eb8940474b63952b6b1bef4843ac"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "62680db35ccf21b945ceb230be12816c"
server_key_iv: "8be548f3b458abd7db89b80aabce2242"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.937105 10315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "a9a6eb8940474b63952b6b1bef4843ac"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "62680db35ccf21b945ceb230be12816c"
server_key_iv: "8be548f3b458abd7db89b80aabce2242"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.942797 10315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:38.946408 10346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:38.947685 10315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:38.947868 10315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "a9a6eb8940474b63952b6b1bef4843ac"
format_stamp: "Formatted at 2026-08-12 16:51:38 on dist-test-slave-z8x9"
server_key: "62680db35ccf21b945ceb230be12816c"
server_key_iv: "8be548f3b458abd7db89b80aabce2242"
server_key_version: "encryptionkey@0"
I20260812 16:51:38.948084 10315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:38.970530 10315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:38.975422 10315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:38.975740 10315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:38.976804 10315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:38.978241 10315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:38.978323 10315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:38.978498 10315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:38.978559 10315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.012414 10315 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:45027
I20260812 16:51:39.012493 10599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:45027 every 8 connection(s)
I20260812 16:51:39.013813 10315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:51:39.015050 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10315
I20260812 16:51:39.015287 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:51:39.016099 8974 external_mini_cluster.cc:1428] Setting key 4842279976e50b936fe4981a9438ab46
Aug 12 16:51:39 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553498, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
I20260812 16:51:39.038932 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:39.016693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34733 (local address 127.8.195.190:39259)
0812 16:51:39.017354 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:39.017361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:39.019043 (+ 1682us) server_negotiation.cc:407] Connection header received
0812 16:51:39.022049 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:39.022054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:39.022134 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:39.022259 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:39.024379 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.025086 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.026915 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.027661 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.027803 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:39.031641 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:39.031650 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:39.031658 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:39.031705 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:39.035485 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:39.036282 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:39.036289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:39.036293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:39.036449 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:39.036911 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:39.036919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:39.036922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:39.037357 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:39.037526 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:39.038249 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 16:51:39.038492 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":329,"threads_started":1}
I20260812 16:51:39.039769 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:39.017910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39259 (local address 127.8.195.129:34733)
0812 16:51:39.018818 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:39.018884 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:39.021725 (+ 2841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:39.022543 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:39.022559 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:39.023376 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:39.024186 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:39.024213 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.025262 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.025278 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:39.026745 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:39.026758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.027879 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.027884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:39.028087 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:39.028820 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:39.028847 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:39.031400 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:39.035656 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:39.035663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:39.035676 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:39.036125 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:39.036569 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:39.036573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:39.036575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:39.036778 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:39.037494 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:39.037501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:39.037834 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 16:51:39.038063 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 16:51:39.038455 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":370,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
I20260812 16:51:39.041366 10600 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39259
I20260812 16:51:39.041682 10600 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:39.042438 10600 heartbeater.cc:511] Master 127.8.195.190:39259 requested a full tablet report, sending...
I20260812 16:51:39.044587 10180 ts_manager.cc:194] Registered new tserver with Master: a9a6eb8940474b63952b6b1bef4843ac (127.8.195.129:45027)
I20260812 16:51:39.046108 10180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:34733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:51:39.066583 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:39.389917 10609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:39.390206 10609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:39.390381 10609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:39.397517 10609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:39.397650 10609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:39.397847 10609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:51:39.407507 10609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:39.408708 10609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:39.410036 10609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:39.423308 10615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:39.423280 10617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:39.423357 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:39.424286 10609 server_base.cc:1034] running on GCE node
I20260812 16:51:39.424881 10609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:39.425865 10609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:39.427089 10609 hybrid_clock.cc:648] HybridClock initialized: now 1786553499427057 us; error 50 us; skew 500 ppm
I20260812 16:51:39.427402 10609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:39 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553499, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:39.430516 10609 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:51:39.432004 10609 webserver.cc:460] Webserver started at http://127.8.195.130:33755/ using document root <none> and password file <none>
I20260812 16:51:39.432694 10609 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:39.432842 10609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:39.433208 10609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:39.436075 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "939df5e5e0fe4458b51ee63c69c3c549"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "5dfd2a05b61f07b1a05112c0c6c7f078"
server_key_iv: "79084522b437dd0dfe7803b3e0d4072b"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.436862 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "939df5e5e0fe4458b51ee63c69c3c549"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "5dfd2a05b61f07b1a05112c0c6c7f078"
server_key_iv: "79084522b437dd0dfe7803b3e0d4072b"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.441962 10609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 16:51:39.445397 10631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.446765 10609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:51:39.446897 10609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "939df5e5e0fe4458b51ee63c69c3c549"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "5dfd2a05b61f07b1a05112c0c6c7f078"
server_key_iv: "79084522b437dd0dfe7803b3e0d4072b"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.447059 10609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:39.463286 10609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:39.467481 10609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:39.467736 10609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:39.468636 10609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:39.470034 10609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:39.470109 10609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.470199 10609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:39.470244 10609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.501659 10609 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:41385
I20260812 16:51:39.501682 10847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:41385 every 8 connection(s)
I20260812 16:51:39.502941 10609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:51:39.505992 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10609
I20260812 16:51:39.506196 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:51:39.506793 8974 external_mini_cluster.cc:1428] Setting key 77d7002f9c352d9b8a7b38eaecedda52
Aug 12 16:51:39 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553499, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:39.525323 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:39.505578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:42563 (local address 127.8.195.190:39259)
0812 16:51:39.505815 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:39.505821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:39.507138 (+ 1317us) server_negotiation.cc:407] Connection header received
0812 16:51:39.509364 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:39.509368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:39.509436 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:39.509525 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:39.511672 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.512414 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.514892 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.515340 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.515442 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:39.519584 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:39.519593 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:39.519602 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:39.519636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:39.522266 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:39.523010 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:39.523016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:39.523021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:39.523156 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:39.523603 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:39.523607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:39.523609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:39.523869 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:39.524024 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:39.524721 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 16:51:39.524960 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 16:51:39.526391 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:39.506065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39259 (local address 127.8.195.130:42563)
0812 16:51:39.506966 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:39.507014 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:39.509053 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:39.509805 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:39.509816 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:39.510623 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:39.511463 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:39.511491 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.512583 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.512591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:39.514722 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:39.514740 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:39.515659 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:39.515668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:39.516026 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:39.516857 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:39.516880 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:39.519358 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:39.522421 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:39.522431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:39.522447 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:39.522879 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:39.523270 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:39.523275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:39.523279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:39.523473 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:39.523997 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:39.524005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:39.524358 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 16:51:39.524623 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 16:51:39.525189 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":339,"threads_started":1}
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
I20260812 16:51:39.527959 10848 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39259
I20260812 16:51:39.528303 10848 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:39.529106 10848 heartbeater.cc:511] Master 127.8.195.190:39259 requested a full tablet report, sending...
I20260812 16:51:39.530694 10180 ts_manager.cc:194] Registered new tserver with Master: 939df5e5e0fe4458b51ee63c69c3c549 (127.8.195.130:41385)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
I20260812 16:51:39.531700 10180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:42563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:51:39.556962 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:39.884567 10859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:39.884927 10859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:39.885128 10859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:39.892298 10859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:39.892405 10859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:39.892594 10859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:51:39.902364 10859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39259
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:39.903656 10859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:39.904971 10859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:39.918100 10871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:39.918125 10873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:39.918182 10870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:39.919251 10859 server_base.cc:1034] running on GCE node
I20260812 16:51:39.919809 10859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:39.920665 10859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:39.921869 10859 hybrid_clock.cc:648] HybridClock initialized: now 1786553499921814 us; error 54 us; skew 500 ppm
I20260812 16:51:39.922137 10859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:39 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553499, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:39.925451 10859 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:51:39.926970 10859 webserver.cc:460] Webserver started at http://127.8.195.131:41631/ using document root <none> and password file <none>
I20260812 16:51:39.927681 10859 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:39.927819 10859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:39.928151 10859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:39.931080 10859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "23e2413406814151b34e17ac4b711072"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "510a34175ee5f8e688589952638deb32"
server_key_iv: "0f9e88512642eb432b0bc532010562a3"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.931871 10859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "23e2413406814151b34e17ac4b711072"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "510a34175ee5f8e688589952638deb32"
server_key_iv: "0f9e88512642eb432b0bc532010562a3"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.936995 10859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:51:39.941640 10886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.942912 10859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:39.943050 10859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "23e2413406814151b34e17ac4b711072"
format_stamp: "Formatted at 2026-08-12 16:51:39 on dist-test-slave-z8x9"
server_key: "510a34175ee5f8e688589952638deb32"
server_key_iv: "0f9e88512642eb432b0bc532010562a3"
server_key_version: "encryptionkey@0"
I20260812 16:51:39.943214 10859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:39.957654 10859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:39.961733 10859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:39.961983 10859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:39.962962 10859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:39.964326 10859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:39.964401 10859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.964485 10859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:39.964540 10859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:39.998582 10859 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:46427
I20260812 16:51:39.998627 11100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:46427 every 8 connection(s)
I20260812 16:51:39.999926 10859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:51:40.007324 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10859
I20260812 16:51:40.007531 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:51:40.008095 8974 external_mini_cluster.cc:1428] Setting key 7b201e3d74cfd2cca272b37849a7c118
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553499, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:40.022882 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.002602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:34463 (local address 127.8.195.190:39259)
0812 16:51:40.002816 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.002821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.003903 (+ 1082us) server_negotiation.cc:407] Connection header received
0812 16:51:40.005915 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.005920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.006023 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.006120 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.007993 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.008803 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.010920 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.011463 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.011566 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.016489 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.016494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.016497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.016531 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.019486 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.020240 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.020245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.020247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.020472 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.020945 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.020950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.020955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.021341 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.021497 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.022226 (+ 729us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.022467 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:51:40.023813 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.003029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39259 (local address 127.8.195.131:34463)
0812 16:51:40.003745 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.003782 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.005622 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.006371 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.006379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.006958 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:40.007808 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.007833 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.008962 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.008966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.010764 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.010777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.011747 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.011754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.012040 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.012852 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:40.012902 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:40.016246 (+ 3344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:40.019646 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:40.019654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:40.019669 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:40.020098 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:40.020617 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:40.020622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:40.020626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:40.020825 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:40.021469 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:40.021477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:40.021817 (+ 340us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.022070 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.022622 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":291,"threads_started":1}
I20260812 16:51:40.025151 11101 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39259
I20260812 16:51:40.025559 11101 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:40.026371 11101 heartbeater.cc:511] Master 127.8.195.190:39259 requested a full tablet report, sending...
I20260812 16:51:40.028007 10180 ts_manager.cc:194] Registered new tserver with Master: 23e2413406814151b34e17ac4b711072 (127.8.195.131:46427)
I20260812 16:51:40.028873 10180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:34463
I20260812 16:51:40.036095 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:51:40.049997 10600 heartbeater.cc:503] Master 127.8.195.190:39259 was elected leader, sending a full tablet report...
I20260812 16:51:40.058966 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.043830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.8.195.190:39259)
0812 16:51:40.043993 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.043998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.044599 (+ 601us) server_negotiation.cc:407] Connection header received
0812 16:51:40.044717 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.044722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.044788 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.044874 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.046907 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.047690 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.049237 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.049687 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.049795 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.053389 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.053394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.053398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.053428 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.056411 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.056998 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.057002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.057004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.057154 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.057553 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.057557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.057558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.057883 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.058074 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.058383 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.058567 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 16:51:40.068779 10180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:40.078218 10180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:40.123778 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.096016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.8.195.131:46427)
0812 16:51:40.109545 (+ 13529us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.109556 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.111304 (+ 1748us) server_negotiation.cc:407] Connection header received
0812 16:51:40.111683 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.111689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.111920 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.112049 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:40.115009 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.117035 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.120156 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.121741 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.121865 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.122156 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.122972 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.123190 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13282,"thread_start_us":12899,"threads_started":1}
I20260812 16:51:40.124369 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.096166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.8.195.130:41385)
0812 16:51:40.109820 (+ 13654us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.109827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.111395 (+ 1568us) server_negotiation.cc:407] Connection header received
0812 16:51:40.114600 (+ 3205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.114611 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.114862 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.115067 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:40.116371 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.118028 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.121000 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.122416 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.122559 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.122745 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.123473 (+ 728us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.123633 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13412,"thread_start_us":13642,"threads_started":1}
I20260812 16:51:40.124372 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.095870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41385 (local address 127.0.0.1:47448)
0812 16:51:40.111057 (+ 15187us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.111133 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.114346 (+ 3213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.115357 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.115363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.115392 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:40.116141 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.116155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.118210 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.118221 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.120839 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.120850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.123057 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.123069 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.123272 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.123295 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.123432 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.123634 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14912,"mutex_wait_us":7,"spinlock_wait_cycles":20864,"thread_start_us":14711,"threads_started":1}
I20260812 16:51:40.124855 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.095468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:46427 (local address 127.0.0.1:53486)
0812 16:51:40.111057 (+ 15589us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.111134 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.111420 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.112354 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.112360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.112384 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:40.114737 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.114751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.117274 (+ 2523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.117279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.119974 (+ 2695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.119991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.122394 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.122402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.122647 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.122709 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.124097 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.124243 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15235,"spinlock_wait_cycles":9216,"thread_start_us":14828,"threads_started":1}
I20260812 16:51:40.128332 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.114464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45027 (local address 127.0.0.1:55490)
0812 16:51:40.118741 (+ 4277us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.118762 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.118895 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.119791 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.119798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.119833 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:40.120276 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.120288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.122281 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.122293 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.125376 (+ 3083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.125404 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.127462 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.127481 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.127668 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.127705 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.127805 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.127929 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4090,"thread_start_us":3925,"threads_started":1}
I20260812 16:51:40.129269 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.111900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.8.195.129:45027)
0812 16:51:40.116288 (+ 4388us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.116298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.119035 (+ 2737us) server_negotiation.cc:407] Connection header received
0812 16:51:40.119148 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.119156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.119418 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.119617 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:40.120455 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.122051 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.125599 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.127150 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.127350 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.127630 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.128244 (+ 614us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.128592 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4132,"thread_start_us":4977,"threads_started":1}
I20260812 16:51:40.129809 10971 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe4875b9044aeaa746778a2fd3367d (DEFAULT_TABLE table=test-table [id=a3065a9d9cde4c768291deea00f05159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:40.131352 10722 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe4875b9044aeaa746778a2fd3367d (DEFAULT_TABLE table=test-table [id=a3065a9d9cde4c768291deea00f05159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:40.131749 10971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe4875b9044aeaa746778a2fd3367d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:40.132510 10722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe4875b9044aeaa746778a2fd3367d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:40.133102 10448 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe4875b9044aeaa746778a2fd3367d (DEFAULT_TABLE table=test-table [id=a3065a9d9cde4c768291deea00f05159]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:40.134550 10448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe4875b9044aeaa746778a2fd3367d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:40.146855 11144 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Bootstrap starting.
I20260812 16:51:40.151641 11144 tablet_bootstrap.cc:654] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:40.153285 11144 log.cc:824] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:40.156806 11145 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Bootstrap starting.
I20260812 16:51:40.159518 11146 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Bootstrap starting.
I20260812 16:51:40.161464 11145 tablet_bootstrap.cc:654] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:40.163093 11145 log.cc:824] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:40.165009 11144 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: No bootstrap required, opened a new log
I20260812 16:51:40.165375 11144 ts_tablet_manager.cc:1397] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.007s
I20260812 16:51:40.165401 11146 tablet_bootstrap.cc:654] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:40.166786 11145 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: No bootstrap required, opened a new log
I20260812 16:51:40.166853 11146 log.cc:824] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:40.167068 11145 ts_tablet_manager.cc:1397] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 16:51:40.170584 11146 tablet_bootstrap.cc:492] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: No bootstrap required, opened a new log
I20260812 16:51:40.170938 11146 ts_tablet_manager.cc:1397] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260812 16:51:40.177654 11144 raft_consensus.cc:348] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.178239 11144 raft_consensus.cc:374] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:40.178354 11144 raft_consensus.cc:729] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939df5e5e0fe4458b51ee63c69c3c549, State: Initialized, Role: FOLLOWER
I20260812 16:51:40.179033 11144 consensus_queue.cc:260] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.179225 11145 raft_consensus.cc:348] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.179695 11145 raft_consensus.cc:374] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:40.179483 11146 raft_consensus.cc:348] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.179831 11145 raft_consensus.cc:729] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e2413406814151b34e17ac4b711072, State: Initialized, Role: FOLLOWER
I20260812 16:51:40.179922 11146 raft_consensus.cc:374] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:40.180028 11146 raft_consensus.cc:729] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a6eb8940474b63952b6b1bef4843ac, State: Initialized, Role: FOLLOWER
I20260812 16:51:40.180548 11146 consensus_queue.cc:260] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.180763 11145 consensus_queue.cc:260] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.184128 11146 ts_tablet_manager.cc:1428] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Time spent starting tablet: real 0.013s user 0.002s sys 0.011s
I20260812 16:51:40.185350 10848 heartbeater.cc:503] Master 127.8.195.190:39259 was elected leader, sending a full tablet report...
I20260812 16:51:40.186245 11144 ts_tablet_manager.cc:1428] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Time spent starting tablet: real 0.021s user 0.015s sys 0.005s
I20260812 16:51:40.187778 11101 heartbeater.cc:503] Master 127.8.195.190:39259 was elected leader, sending a full tablet report...
I20260812 16:51:40.191018 11145 ts_tablet_manager.cc:1428] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Time spent starting tablet: real 0.024s user 0.022s sys 0.000s
W20260812 16:51:40.251720 11102 tablet.cc:2369] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:51:40.255043 10853 tablet.cc:2369] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:51:40.266254 10601 tablet.cc:2369] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:51:40.269556 11152 raft_consensus.cc:482] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:40.269879 11152 raft_consensus.cc:504] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.272135 11152 leader_election.cc:288] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939df5e5e0fe4458b51ee63c69c3c549 (127.8.195.130:41385), a9a6eb8940474b63952b6b1bef4843ac (127.8.195.129:45027)
I20260812 16:51:40.278733 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.272774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:52903 (local address 127.8.195.129:45027)
0812 16:51:40.273001 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.273008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.273029 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:51:40.273116 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.273120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.273230 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.273331 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:40.273736 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.274480 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.276594 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.277666 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.277895 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.278001 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.278306 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.278436 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:51:40.278877 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.272494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45027 (local address 127.8.195.131:52903)
0812 16:51:40.272818 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.272841 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.272977 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.273356 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.273360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.273388 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:40.273587 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.273596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.274618 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.274624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.276421 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.276434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.277827 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.277835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.278126 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.278152 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.278274 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.278422 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260812 16:51:40.279982 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.272530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:38327 (local address 127.8.195.130:41385)
0812 16:51:40.272744 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.272753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.273524 (+ 771us) server_negotiation.cc:407] Connection header received
0812 16:51:40.273730 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.273735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.273802 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.273921 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:40.275094 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.275801 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.277757 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.278618 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.278833 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.278943 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.279547 (+ 604us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.279676 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:51:40.280395 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.272549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41385 (local address 127.8.195.131:38327)
0812 16:51:40.273403 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:40.273418 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:40.273578 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:40.273937 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:40.273941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:40.273964 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:40.274844 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.274856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.275954 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.275960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.277603 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:40.277615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.279221 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.279230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:40.279379 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.279404 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 16:51:40.279514 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 16:51:40.279640 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":496,"threads_started":1}
I20260812 16:51:40.280342 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe4875b9044aeaa746778a2fd3367d" candidate_uuid: "23e2413406814151b34e17ac4b711072" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a6eb8940474b63952b6b1bef4843ac" is_pre_election: true
I20260812 16:51:40.281386 10490 raft_consensus.cc:2454] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23e2413406814151b34e17ac4b711072 in term 0.
I20260812 16:51:40.281788 10770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe4875b9044aeaa746778a2fd3367d" candidate_uuid: "23e2413406814151b34e17ac4b711072" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939df5e5e0fe4458b51ee63c69c3c549" is_pre_election: true
I20260812 16:51:40.282397 10891 leader_election.cc:302] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e2413406814151b34e17ac4b711072, a9a6eb8940474b63952b6b1bef4843ac; no voters:
I20260812 16:51:40.282449 10770 raft_consensus.cc:2454] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23e2413406814151b34e17ac4b711072 in term 0.
I20260812 16:51:40.282985 11152 raft_consensus.cc:2781] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:40.283149 11152 raft_consensus.cc:482] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:40.283291 11152 raft_consensus.cc:3037] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:40.285810 11152 raft_consensus.cc:504] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.286589 11152 leader_election.cc:288] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [CANDIDATE]: Term 1 election: Requested vote from peers 939df5e5e0fe4458b51ee63c69c3c549 (127.8.195.130:41385), a9a6eb8940474b63952b6b1bef4843ac (127.8.195.129:45027)
I20260812 16:51:40.286963 10770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe4875b9044aeaa746778a2fd3367d" candidate_uuid: "23e2413406814151b34e17ac4b711072" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939df5e5e0fe4458b51ee63c69c3c549"
I20260812 16:51:40.287060 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe4875b9044aeaa746778a2fd3367d" candidate_uuid: "23e2413406814151b34e17ac4b711072" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a6eb8940474b63952b6b1bef4843ac"
I20260812 16:51:40.287194 10770 raft_consensus.cc:3037] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:40.287259 10490 raft_consensus.cc:3037] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:40.290623 10770 raft_consensus.cc:2454] T 4ebe4875b9044aeaa746778a2fd3367d P 939df5e5e0fe4458b51ee63c69c3c549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23e2413406814151b34e17ac4b711072 in term 1.
I20260812 16:51:40.290619 10490 raft_consensus.cc:2454] T 4ebe4875b9044aeaa746778a2fd3367d P a9a6eb8940474b63952b6b1bef4843ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23e2413406814151b34e17ac4b711072 in term 1.
I20260812 16:51:40.291235 10891 leader_election.cc:302] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e2413406814151b34e17ac4b711072, a9a6eb8940474b63952b6b1bef4843ac; no voters:
I20260812 16:51:40.291577 11152 raft_consensus.cc:2781] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:40.292091 11152 raft_consensus.cc:686] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [term 1 LEADER]: Becoming Leader. State: Replica: 23e2413406814151b34e17ac4b711072, State: Running, Role: LEADER
I20260812 16:51:40.292608 11152 consensus_queue.cc:237] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } }
I20260812 16:51:40.298394 10176 catalog_manager.cc:5495] T 4ebe4875b9044aeaa746778a2fd3367d P 23e2413406814151b34e17ac4b711072 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23e2413406814151b34e17ac4b711072 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "23e2413406814151b34e17ac4b711072" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23e2413406814151b34e17ac4b711072" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 46427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939df5e5e0fe4458b51ee63c69c3c549" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9a6eb8940474b63952b6b1bef4843ac" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45027 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:40.337738 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.320248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55498 (local address 127.8.195.129:45027)
0812 16:51:40.320425 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.320430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.321140 (+ 710us) server_negotiation.cc:407] Connection header received
0812 16:51:40.321406 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.321411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.321520 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.321638 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.323576 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.324447 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.325888 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.326320 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.326470 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.331281 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.331294 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.331309 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.331379 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.334723 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.335587 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.335595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.335597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.335771 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.336108 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.336112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.336114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.336506 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.336743 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.337075 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.337386 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:40.394884 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.378751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.8.195.129:45027)
0812 16:51:40.378961 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.378969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.379441 (+ 472us) server_negotiation.cc:407] Connection header received
0812 16:51:40.379653 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.379660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.379740 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.379846 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.381420 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.382195 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.383526 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.384029 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.384187 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.388714 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.388720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.388725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.388765 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.391825 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.392573 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.392588 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.392591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.392737 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.393292 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.393299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.393303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.393598 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.393911 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.394208 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.394497 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:40.452498 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.435873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.8.195.129:45027)
0812 16:51:40.436116 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.436123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.436417 (+ 294us) server_negotiation.cc:407] Connection header received
0812 16:51:40.436551 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.436555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.436624 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.436719 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.438631 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.439530 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.441043 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.441674 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.441773 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.445887 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.445893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.445897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.445934 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.449060 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.450033 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.450039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.450043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.450314 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.450780 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.450785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.450789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.451130 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.451418 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.451784 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.452062 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 16:51:40.454715 10407 tablet_service.cc:2908] Rejecting scan request for tablet 4ebe4875b9044aeaa746778a2fd3367d: Uninitialized: clean time has not yet been initialized
I20260812 16:51:40.459036 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10315
I20260812 16:51:40.473431 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.459960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.8.195.129:45027)
0812 16:51:40.460194 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.460204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.460506 (+ 302us) server_negotiation.cc:407] Connection header received
0812 16:51:40.460621 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.460630 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.460701 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.460816 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.462030 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.462787 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.464067 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.464599 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.464777 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.467789 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.467795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.467798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.467831 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.470978 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.471597 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.471601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.471603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.471738 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.472084 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.472089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.472091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.472399 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.472666 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.472859 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.473030 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:51:40.474260 10586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:40.564359 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10315
I20260812 16:51:40.591897 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10609
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:51:40.607785 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.592777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.8.195.130:41385)
0812 16:51:40.592990 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.592998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.593023 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:51:40.593281 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.593285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.593371 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.593500 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.594784 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.595604 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.596942 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.597502 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.597648 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.601770 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.601786 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.601804 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.601838 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.604779 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.605589 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.605597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.605600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.605748 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.606147 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.606151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.606153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.606558 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.606918 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.607214 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.607451 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:51:40.608494 10838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:40.701491 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10609
I20260812 16:51:40.728008 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10859
Aug 12 16:51:40 dist-test-slave-z8x9 krb5kdc[10079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:51:40.743552 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:40.728824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.8.195.131:46427)
0812 16:51:40.729333 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:40.729338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:40.729355 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:51:40.729412 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:40.729416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:40.729476 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:40.729570 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:40.730938 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.731938 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.733431 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:40.733841 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:40.733947 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:40.738013 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:40.738029 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:40.738046 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:40.738088 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:40.740832 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.741734 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.741741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.741743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.741894 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:40.742225 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:40.742228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:40.742230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:40.742589 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:40.742813 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:40.743054 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 16:51:40.743227 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":266,"threads_started":1}
I20260812 16:51:40.744269 11094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:40.833616 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10859
I20260812 16:51:40.862336 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10122
I20260812 16:51:40.863680 10248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:40.975745 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10122
2026-08-12T16:51:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4596 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:51:41 dist-test-slave-z8x9 krb5kdc[11201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:51:41 dist-test-slave-z8x9 krb5kdc[11201](info): set up 2 sockets
Aug 12 16:51:41 dist-test-slave-z8x9 krb5kdc[11201](info): commencing operation
krb5kdc: starting...
W20260812 16:51:42.355616 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:51:42 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:51:42.573108 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37715
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:42.906215 11227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:42.906584 11227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:42.906750 11227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:42.914104 11227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:51:42.914247 11227 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:51:42.914384 11227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:42.914505 11227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:42.914597 11227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:51:42.914687 11227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:51:42.924583 11227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37715
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:42.925931 11227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:42.927245 11227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:42.939293 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:42.939214 11241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:42.939235 11237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:42.940462 11227 server_base.cc:1034] running on GCE node
I20260812 16:51:42.941401 11227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:42.942910 11227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:42.944165 11227 hybrid_clock.cc:648] HybridClock initialized: now 1786553502944094 us; error 48 us; skew 500 ppm
I20260812 16:51:42.944506 11227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:42 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:42.947902 11227 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:51:42.949456 11227 webserver.cc:460] Webserver started at http://127.8.195.190:43967/ using document root <none> and password file <none>
I20260812 16:51:42.950393 11227 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:42.950522 11227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:42.950896 11227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:42.954624 11227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "023c1c68868f423ebcabf91e4a0605b7"
format_stamp: "Formatted at 2026-08-12 16:51:42 on dist-test-slave-z8x9"
server_key: "e20dd5abb22b6eeda6df620bc31df6e0"
server_key_iv: "d14a20c2992db3bc8e3a20ec0241e3d1"
server_key_version: "encryptionkey@0"
I20260812 16:51:42.955492 11227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "023c1c68868f423ebcabf91e4a0605b7"
format_stamp: "Formatted at 2026-08-12 16:51:42 on dist-test-slave-z8x9"
server_key: "e20dd5abb22b6eeda6df620bc31df6e0"
server_key_iv: "d14a20c2992db3bc8e3a20ec0241e3d1"
server_key_version: "encryptionkey@0"
I20260812 16:51:42.960917 11227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 16:51:42.964437 11250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:42.965878 11227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:51:42.966079 11227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "023c1c68868f423ebcabf91e4a0605b7"
format_stamp: "Formatted at 2026-08-12 16:51:42 on dist-test-slave-z8x9"
server_key: "e20dd5abb22b6eeda6df620bc31df6e0"
server_key_iv: "d14a20c2992db3bc8e3a20ec0241e3d1"
server_key_version: "encryptionkey@0"
I20260812 16:51:42.966310 11227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:42.983701 11227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:42.988546 11227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:42.988857 11227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:43.011673 11227 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37715
I20260812 16:51:43.011698 11392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37715 every 8 connection(s)
I20260812 16:51:43.013087 11227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:51:43.014312 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11227
I20260812 16:51:43.014559 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:51:43.015406 8974 external_mini_cluster.cc:1428] Setting key c827ff81980144c78cf54821e937dcca
I20260812 16:51:43.018046 11393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:51:43 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:43.031581 11393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Bootstrap starting.
I20260812 16:51:43.035842 11393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:43.037075 11393 log.cc:824] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:43.040608 11393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: No bootstrap required, opened a new log
I20260812 16:51:43.041131 11400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:43.017635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.8.195.190:37715)
0812 16:51:43.018659 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:43.018673 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:43.018717 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:51:43.020407 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:43.020429 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:43.020814 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:43.021141 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:43.023029 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.024433 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.026199 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.026854 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.026988 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:43.031887 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:43.031901 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:43.031923 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:43.031975 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:43.036505 (+ 4530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.037324 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.037333 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.037340 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.037540 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.038053 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.038057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.038059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.038671 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:43.038974 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:43.039357 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 16:51:43.039787 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":505,"threads_started":1}
I20260812 16:51:43.048055 11393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } }
I20260812 16:51:43.048394 11393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:43.048470 11393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 023c1c68868f423ebcabf91e4a0605b7, State: Initialized, Role: FOLLOWER
I20260812 16:51:43.049003 11393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } }
I20260812 16:51:43.049273 11393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:51:43.049436 11393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:51:43.049612 11393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:43.051771 11393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } }
I20260812 16:51:43.052309 11393 leader_election.cc:302] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 023c1c68868f423ebcabf91e4a0605b7; no voters:
I20260812 16:51:43.052934 11393 leader_election.cc:288] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:51:43.053097 11404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:43.053946 11404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [term 1 LEADER]: Becoming Leader. State: Replica: 023c1c68868f423ebcabf91e4a0605b7, State: Running, Role: LEADER
I20260812 16:51:43.054554 11404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } }
I20260812 16:51:43.055120 11393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:51:43.057232 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "023c1c68868f423ebcabf91e4a0605b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } } }
I20260812 16:51:43.057636 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:43.061462 11406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 023c1c68868f423ebcabf91e4a0605b7. Latest consensus state: current_term: 1 leader_uuid: "023c1c68868f423ebcabf91e4a0605b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023c1c68868f423ebcabf91e4a0605b7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37715 } } }
I20260812 16:51:43.061862 11406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:43.062624 11419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:51:43.068069 11419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:51:43.077894 11419 catalog_manager.cc:1347] Generated new cluster ID: 36f4dc8ef56146cfa062f447c5079d82
I20260812 16:51:43.078035 11419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:51:43.107362 11419 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:51:43.108834 11419 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:51:43.115190 11419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Generated new TSK 0
I20260812 16:51:43.116075 11419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:51:43.173493 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37715
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:43.522171 11445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:43.522470 11445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:43.522660 11445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:43.529866 11445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:43.530017 11445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:43.530171 11445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:43.530318 11445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:51:43.541006 11445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:43.542382 11445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:43.543802 11445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:43.558768 11455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:43.558907 11464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:43.559106 11453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:43.564867 11445 server_base.cc:1034] running on GCE node
I20260812 16:51:43.565784 11445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:43.567412 11445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:43.568714 11445 hybrid_clock.cc:648] HybridClock initialized: now 1786553503568578 us; error 103 us; skew 500 ppm
I20260812 16:51:43.569136 11445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:43 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:43.573979 11445 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:51:43.578052 11445 webserver.cc:460] Webserver started at http://127.8.195.129:43489/ using document root <none> and password file <none>
I20260812 16:51:43.579138 11445 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:43.579288 11445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:43.579718 11445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:43.583806 11445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "256238c37f174e899b11baf28176ceef"
format_stamp: "Formatted at 2026-08-12 16:51:43 on dist-test-slave-z8x9"
server_key: "74169e343779ae5760615f98f35c12f7"
server_key_iv: "9d8e855006b1a912c4f41abbfd24c505"
server_key_version: "encryptionkey@0"
I20260812 16:51:43.584883 11445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "256238c37f174e899b11baf28176ceef"
format_stamp: "Formatted at 2026-08-12 16:51:43 on dist-test-slave-z8x9"
server_key: "74169e343779ae5760615f98f35c12f7"
server_key_iv: "9d8e855006b1a912c4f41abbfd24c505"
server_key_version: "encryptionkey@0"
I20260812 16:51:43.591557 11445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260812 16:51:43.596481 11475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:43.598162 11445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:43.598336 11445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "256238c37f174e899b11baf28176ceef"
format_stamp: "Formatted at 2026-08-12 16:51:43 on dist-test-slave-z8x9"
server_key: "74169e343779ae5760615f98f35c12f7"
server_key_iv: "9d8e855006b1a912c4f41abbfd24c505"
server_key_version: "encryptionkey@0"
I20260812 16:51:43.598579 11445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:43.615296 11445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:43.619685 11445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:43.620020 11445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:43.622851 11445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:43.622995 11445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:43.623168 11445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:43.623265 11445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:43 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:43.648267 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:43.623746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:60103 (local address 127.8.195.190:37715)
0812 16:51:43.624286 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:43.624293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:43.625532 (+ 1239us) server_negotiation.cc:407] Connection header received
0812 16:51:43.627875 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:43.627884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:43.627990 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:43.628085 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:43.630725 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.632050 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.634721 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.635129 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.635282 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:43.640078 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:43.640083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:43.640087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:43.640121 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:43.643747 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.644767 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.644774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.644777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.644947 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.645604 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.645613 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.645616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.646027 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:43.646188 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:43.647341 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0812 16:51:43.647700 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":258,"threads_started":1}
I20260812 16:51:43.648972 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:43.624417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.129:60103)
0812 16:51:43.625337 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:43.625396 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:43.627407 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:43.628668 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:43.628682 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:43.629576 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:43.630458 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:43.630496 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.632276 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.632285 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:43.634531 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:43.634556 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.635393 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.635402 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:43.635612 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:43.636427 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:43.636450 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:43.639793 (+ 3343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:43.643900 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:43.643909 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:43.643926 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:43.644601 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:43.645111 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:43.645117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:43.645126 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:43.645459 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:43.646181 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:43.646190 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:43.646750 (+ 560us) client_negotiation.cc:769] Sending connection context
0812 16:51:43.647167 (+ 417us) client_negotiation.cc:240] Negotiation successful
0812 16:51:43.647622 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":391,"threads_started":1}
I20260812 16:51:43.676105 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:43.655504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34781 (local address 127.8.195.190:37715)
0812 16:51:43.655746 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:43.655752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:43.660434 (+ 4682us) server_negotiation.cc:407] Connection header received
0812 16:51:43.660765 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:43.660772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:43.660867 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:43.661001 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:43.662881 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.664063 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.666567 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.667199 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.667344 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:43.669845 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:43.669851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:43.669854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:43.669914 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:43.672883 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.673817 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.673823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.673827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.674007 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:43.674465 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:43.674470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:43.674474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:43.674910 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:43.675099 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:43.675518 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 16:51:43.675739 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:51:43.676431 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:43.655572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.129:34781)
0812 16:51:43.660287 (+ 4715us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:43.660315 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:43.660502 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:43.661063 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:43.661071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:43.661954 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:43.662616 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:43.662636 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.664249 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.664254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:43.666360 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:43.666379 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:43.667429 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:43.667435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:43.667654 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:43.668465 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:43.668492 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:43.669653 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:43.673029 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:43.673036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:43.673046 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:43.673623 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:43.674156 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:43.674162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:43.674166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:43.674337 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:43.675040 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:43.675045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:43.675254 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 16:51:43.675406 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 16:51:43.675809 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4496,"thread_start_us":4306,"threads_started":1}
I20260812 16:51:43.679113 11445 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:33859
I20260812 16:51:43.679196 11698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:33859 every 8 connection(s)
I20260812 16:51:43.680531 11445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:51:43.682755 11699 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37715
I20260812 16:51:43.683010 11699 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:43.683658 11699 heartbeater.cc:511] Master 127.8.195.190:37715 requested a full tablet report, sending...
I20260812 16:51:43.685645 11285 ts_manager.cc:194] Registered new tserver with Master: 256238c37f174e899b11baf28176ceef (127.8.195.129:33859)
I20260812 16:51:43.687193 11285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:60103
I20260812 16:51:43.689097 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11445
I20260812 16:51:43.689340 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:51:43.689991 8974 external_mini_cluster.cc:1428] Setting key 5e3cb41e1d53847d4a4b75b2d97638dd
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:51:43.736743 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37715
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:44.063234 11704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:44.063524 11704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:44.063683 11704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:44.071122 11704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:44.071242 11704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:44.071385 11704 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:44.071507 11704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:51:44.081300 11704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:44.082588 11704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:44.083963 11704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:44.097895 11715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:44.097878 11717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:44.098126 11714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:44.098775 11704 server_base.cc:1034] running on GCE node
I20260812 16:51:44.099360 11704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:44.100272 11704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:44.101491 11704 hybrid_clock.cc:648] HybridClock initialized: now 1786553504101425 us; error 49 us; skew 500 ppm
I20260812 16:51:44.101809 11704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:44.105150 11704 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:51:44.107203 11704 webserver.cc:460] Webserver started at http://127.8.195.130:43967/ using document root <none> and password file <none>
I20260812 16:51:44.107889 11704 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:44.107978 11704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:44.108250 11704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:44.111248 11704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "da465ab6092746bba4d10c2500b1e7ca"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "602d910d5e6f0562d55b610dafad186a"
server_key_iv: "886ab02f749e8a011e48acee70cf0973"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.112035 11704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "da465ab6092746bba4d10c2500b1e7ca"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "602d910d5e6f0562d55b610dafad186a"
server_key_iv: "886ab02f749e8a011e48acee70cf0973"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.117791 11704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 16:51:44.121157 11734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:44.122821 11704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:44.123042 11704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "da465ab6092746bba4d10c2500b1e7ca"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "602d910d5e6f0562d55b610dafad186a"
server_key_iv: "886ab02f749e8a011e48acee70cf0973"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.123320 11704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:44.145344 11704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:44.150008 11704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:44.150324 11704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:44.153367 11704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:44.153486 11704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:44.153625 11704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:44.153703 11704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:44.181617 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.154031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:45841 (local address 127.8.195.190:37715)
0812 16:51:44.154249 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.154257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.156143 (+ 1886us) server_negotiation.cc:407] Connection header received
0812 16:51:44.158469 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.158475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.158589 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.158704 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:44.161631 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.162457 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.165770 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.166375 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.166528 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.172534 (+ 6006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:44.172543 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:44.172551 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:44.172602 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:44.176850 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.178210 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.178218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.178222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.178401 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.179048 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.179054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.179060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.179501 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:44.179693 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.180869 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.181144 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:51:44.184047 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.154989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.130:45841)
0812 16:51:44.155984 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.156023 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.158167 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.159130 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.159144 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.160151 (+ 1007us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:44.161394 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.161428 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.162629 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.162641 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.165589 (+ 2948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.165609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.166790 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.166804 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.167246 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.168139 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:44.168186 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:44.172206 (+ 4020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:44.177021 (+ 4815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.177037 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.177059 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.178036 (+ 977us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.178547 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.178555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.178562 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.178893 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.179669 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:44.179681 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:44.180314 (+ 633us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.180724 (+ 410us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.182043 (+ 1319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":400,"threads_started":1}
I20260812 16:51:44.196798 11704 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40761
I20260812 16:51:44.196936 11977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40761 every 8 connection(s)
I20260812 16:51:44.198125 11704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:51:44.200343 11981 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37715
I20260812 16:51:44.200590 11981 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:44.201438 11981 heartbeater.cc:511] Master 127.8.195.190:37715 requested a full tablet report, sending...
I20260812 16:51:44.202931 11285 ts_manager.cc:194] Registered new tserver with Master: da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761)
I20260812 16:51:44.204144 11285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:45841
I20260812 16:51:44.207420 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.192235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.130:45133)
0812 16:51:44.193112 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.193140 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.193376 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.193763 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.193768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.194231 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:44.194520 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.194529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.195687 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.195692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.197364 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.197381 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.198140 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.198147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.198354 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.199118 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:44.199140 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:44.200961 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:44.204853 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.204859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.204862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.205381 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.205798 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.205801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.205803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.205925 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.206613 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:44.206617 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:44.206784 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.206891 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.207097 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":541,"threads_started":1}
I20260812 16:51:44.207429 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.192494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:45133 (local address 127.8.195.190:37715)
0812 16:51:44.192736 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.192744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.193275 (+ 531us) server_negotiation.cc:407] Connection header received
0812 16:51:44.193539 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.193545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.193622 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.193726 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:44.194671 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.195517 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.197499 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.197936 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.198073 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.201142 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:44.201147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:44.201150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:44.201229 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:44.204722 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.205514 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.205520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.205523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.205681 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.206049 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.206054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.206057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.206481 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:44.206629 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.207040 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.207208 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 16:51:44.207935 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11704
I20260812 16:51:44.208139 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:51:44.208714 8974 external_mini_cluster.cc:1428] Setting key 4a07bb2774452f48ff714b2785873240
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:51:44.256140 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37715
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:44.586658 11992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:44.586941 11992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:44.587124 11992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:44.594475 11992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:44.594583 11992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:44.594718 11992 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:44.594844 11992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:51:44.604741 11992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:37715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:44.606093 11992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:44.607406 11992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:44.620260 12007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:44.620348 12014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:44.620287 12010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:44.621114 11992 server_base.cc:1034] running on GCE node
I20260812 16:51:44.621819 11992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:44.622721 11992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:44.623966 11992 hybrid_clock.cc:648] HybridClock initialized: now 1786553504623896 us; error 47 us; skew 500 ppm
I20260812 16:51:44.624264 11992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:44.627487 11992 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:51:44.629103 11992 webserver.cc:460] Webserver started at http://127.8.195.131:33589/ using document root <none> and password file <none>
I20260812 16:51:44.629825 11992 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:44.629961 11992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:44.630295 11992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:44.633059 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "74a2008db63741cc8d6b0316208d4624"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "c80b5a40d9727e65baec45cee1fc6244"
server_key_iv: "9b07a870ea00db00cf0f8d69666828c0"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.633973 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "74a2008db63741cc8d6b0316208d4624"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "c80b5a40d9727e65baec45cee1fc6244"
server_key_iv: "9b07a870ea00db00cf0f8d69666828c0"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.638842 11992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:51:44.642087 12032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:44.643384 11992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:44.643570 11992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "74a2008db63741cc8d6b0316208d4624"
format_stamp: "Formatted at 2026-08-12 16:51:44 on dist-test-slave-z8x9"
server_key: "c80b5a40d9727e65baec45cee1fc6244"
server_key_iv: "9b07a870ea00db00cf0f8d69666828c0"
server_key_version: "encryptionkey@0"
I20260812 16:51:44.643810 11992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:44.689795 11699 heartbeater.cc:503] Master 127.8.195.190:37715 was elected leader, sending a full tablet report...
I20260812 16:51:44.701474 11992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:44.706158 11992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:44.706478 11992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:44.709578 11992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:44.709692 11992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:44.709836 11992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:44.709900 11992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:44.733628 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.710357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:42907 (local address 127.8.195.190:37715)
0812 16:51:44.710951 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.710956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.712254 (+ 1298us) server_negotiation.cc:407] Connection header received
0812 16:51:44.714448 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.714454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.714545 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.714669 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:44.717058 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.718172 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.720710 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.721589 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.721772 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.727208 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:44.727213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:44.727217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:44.727256 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:44.730434 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.731177 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.731185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.731189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.731334 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.731783 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.731789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.731793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.732067 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:44.732213 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.732962 (+ 749us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.733220 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":376,"threads_started":1}
I20260812 16:51:44.735167 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.711237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.131:42907)
0812 16:51:44.712086 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.712125 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.714161 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.715030 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.715042 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.715930 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:44.716769 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.716815 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.718400 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.718404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.720528 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.720541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.721913 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.721926 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.722248 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.723185 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:44.723215 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:44.726945 (+ 3730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:44.730587 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.730596 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.730612 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.731039 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.731454 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.731460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.731463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.731664 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.732211 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:44.732220 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:44.732572 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.732868 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.733488 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":327,"threads_started":1}
I20260812 16:51:44.756146 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.741345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.8.195.131:51363)
0812 16:51:44.742104 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.742127 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.742292 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.742622 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.742626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.743086 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:44.743387 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.743396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.744387 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.744392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.745884 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.745907 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.746944 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.746954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.747281 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.748156 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:44.748196 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:44.749863 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:44.753591 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.753597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.753601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.754028 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.754420 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:44.754425 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:44.754428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:44.754566 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:44.755131 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:44.755137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:44.755342 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.755532 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.755782 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":454,"threads_started":1}
I20260812 16:51:44.756840 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.741302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:51363 (local address 127.8.195.190:37715)
0812 16:51:44.741505 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.741512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.742219 (+ 707us) server_negotiation.cc:407] Connection header received
0812 16:51:44.742416 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.742423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.742491 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.742595 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:44.743540 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.744229 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.746097 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.746713 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.746883 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.750147 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:44.750155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:44.750160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:44.750207 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:44.753451 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.754157 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.754162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.754166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.754303 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.754713 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.754718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.754720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.755005 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:44.755172 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.755745 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.756080 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:51:44.774278 11992 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:42611
I20260812 16:51:44.774326 12275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:42611 every 8 connection(s)
I20260812 16:51:44.775638 11992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:51:44.778002 12277 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37715
I20260812 16:51:44.778273 12277 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:44.778273 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11992
I20260812 16:51:44.778492 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:51:44.778949 8974 external_mini_cluster.cc:1428] Setting key e221706af358544f90c66fe4cbd6486e
I20260812 16:51:44.779085 12277 heartbeater.cc:511] Master 127.8.195.190:37715 requested a full tablet report, sending...
I20260812 16:51:44.780694 11285 ts_manager.cc:194] Registered new tserver with Master: 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611)
I20260812 16:51:44.781638 11285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:42907
I20260812 16:51:44.782490 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:51:44 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:44.836545 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.822232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.8.195.190:37715)
0812 16:51:44.822496 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.822502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.822693 (+ 191us) server_negotiation.cc:407] Connection header received
0812 16:51:44.822961 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.822966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.823032 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.823143 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:44.824532 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.825365 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.826685 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.827220 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.827373 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.831331 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:44.831336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:44.831345 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:44.831379 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:44.833944 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.834570 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.834578 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.834585 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.834721 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:44.835109 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:44.835115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:44.835120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:44.835401 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:44.835668 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.835972 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.836203 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:51:44.841276 11284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:44.848191 11284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:44.873512 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.863432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.8.195.130:40761)
0812 16:51:44.864079 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.864088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.864117 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:51:44.864283 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.864295 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.864567 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.865546 (+ 979us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:44.865768 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.867098 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.870442 (+ 3344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.871965 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.872127 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.872344 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.872982 (+ 638us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.873162 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":377,"threads_started":1}
I20260812 16:51:44.873612 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.862871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40761 (local address 127.0.0.1:34510)
0812 16:51:44.863452 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.863469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.863585 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.864806 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.864813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.864854 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:44.865329 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.865344 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.867220 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.867225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.870271 (+ 3046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.870289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.872485 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.872491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.872718 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.872771 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.872913 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.873105 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":49,"thread_start_us":775,"threads_started":1}
I20260812 16:51:44.874511 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.861052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.8.195.131:42611)
0812 16:51:44.865959 (+ 4907us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.865966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.865994 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:51:44.866083 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.866089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.866284 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.866476 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:44.867114 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.868719 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.871519 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.872910 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.873037 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.873928 (+ 891us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.873997 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.874164 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4732,"thread_start_us":4564,"threads_started":1}
I20260812 16:51:44.875018 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.861560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.8.195.129:33859)
0812 16:51:44.864653 (+ 3093us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.864664 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.864691 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:51:44.864789 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.864797 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.865015 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.865245 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:44.865948 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.867320 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.872449 (+ 5129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.873929 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.874133 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.874340 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.874578 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.874730 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2934,"thread_start_us":2808,"threads_started":1}
I20260812 16:51:44.875195 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.860473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:42611 (local address 127.0.0.1:48478)
0812 16:51:44.861727 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.861786 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.861990 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.866489 (+ 4499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.866494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.866520 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:44.866893 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.866905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.868884 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.868893 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.871328 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.871347 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.873343 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.873350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.873544 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.873591 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.873686 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.874805 (+ 1119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":2388,"threads_started":1}
I20260812 16:51:44.875577 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.860932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33859 (local address 127.0.0.1:49438)
0812 16:51:44.863161 (+ 2229us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.863184 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.863320 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.865256 (+ 1936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.865263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.865310 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:44.865773 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.865785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.867490 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.867496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.870287 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.870297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.874189 (+ 3892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.874197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.874377 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.874408 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.874503 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.874816 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2086,"mutex_wait_us":84,"thread_start_us":359,"threads_started":1}
I20260812 16:51:44.877756 11853 tablet_service.cc:1467] Processing CreateTablet for tablet e5421beeb8994376b9e09f6a44a4163e (DEFAULT_TABLE table=test-table [id=a0c13aab5415409494563d445a70715f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:44.878464 12150 tablet_service.cc:1467] Processing CreateTablet for tablet e5421beeb8994376b9e09f6a44a4163e (DEFAULT_TABLE table=test-table [id=a0c13aab5415409494563d445a70715f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:44.879129 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5421beeb8994376b9e09f6a44a4163e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:44.879846 12150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5421beeb8994376b9e09f6a44a4163e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:44.880542 11561 tablet_service.cc:1467] Processing CreateTablet for tablet e5421beeb8994376b9e09f6a44a4163e (DEFAULT_TABLE table=test-table [id=a0c13aab5415409494563d445a70715f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:44.881850 11561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5421beeb8994376b9e09f6a44a4163e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:44.891765 12316 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Bootstrap starting.
I20260812 16:51:44.894248 12317 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Bootstrap starting.
I20260812 16:51:44.895239 12316 tablet_bootstrap.cc:654] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:44.895989 12318 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Bootstrap starting.
I20260812 16:51:44.897377 12316 log.cc:824] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:44.898727 12317 tablet_bootstrap.cc:654] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:44.899471 12318 tablet_bootstrap.cc:654] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:44.900084 12317 log.cc:824] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:44.901023 12318 log.cc:824] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:44.903098 12316 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: No bootstrap required, opened a new log
I20260812 16:51:44.903383 12316 ts_tablet_manager.cc:1397] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260812 16:51:44.903561 12317 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: No bootstrap required, opened a new log
I20260812 16:51:44.903839 12317 ts_tablet_manager.cc:1397] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 16:51:44.907153 12318 tablet_bootstrap.cc:492] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: No bootstrap required, opened a new log
I20260812 16:51:44.907414 12318 ts_tablet_manager.cc:1397] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260812 16:51:44.915350 12318 raft_consensus.cc:348] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.915745 12318 raft_consensus.cc:374] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:44.915870 12318 raft_consensus.cc:729] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256238c37f174e899b11baf28176ceef, State: Initialized, Role: FOLLOWER
I20260812 16:51:44.916025 12317 raft_consensus.cc:348] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.916555 12317 raft_consensus.cc:374] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:44.916535 12318 consensus_queue.cc:260] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.916729 12317 raft_consensus.cc:729] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a2008db63741cc8d6b0316208d4624, State: Initialized, Role: FOLLOWER
I20260812 16:51:44.916769 12316 raft_consensus.cc:348] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.917310 12316 raft_consensus.cc:374] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:44.917413 12316 raft_consensus.cc:729] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da465ab6092746bba4d10c2500b1e7ca, State: Initialized, Role: FOLLOWER
I20260812 16:51:44.917575 12317 consensus_queue.cc:260] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.918061 12316 consensus_queue.cc:260] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.919034 12277 heartbeater.cc:503] Master 127.8.195.190:37715 was elected leader, sending a full tablet report...
I20260812 16:51:44.919812 12317 ts_tablet_manager.cc:1428] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260812 16:51:44.924872 12318 ts_tablet_manager.cc:1428] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Time spent starting tablet: real 0.017s user 0.005s sys 0.012s
I20260812 16:51:44.926013 11981 heartbeater.cc:503] Master 127.8.195.190:37715 was elected leader, sending a full tablet report...
I20260812 16:51:44.926988 12316 ts_tablet_manager.cc:1428] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Time spent starting tablet: real 0.023s user 0.016s sys 0.007s
W20260812 16:51:44.932982 11701 tablet.cc:2369] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:51:44.950393 11985 tablet.cc:2369] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:51:44.969000 12329 raft_consensus.cc:482] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:44.969455 12329 raft_consensus.cc:504] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.971596 12329 leader_election.cc:288] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 256238c37f174e899b11baf28176ceef (127.8.195.129:33859), da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761)
I20260812 16:51:44.979199 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.972095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33859 (local address 127.8.195.131:52617)
0812 16:51:44.972336 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.972358 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.972498 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.972978 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.972983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.973017 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:44.973320 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.973333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.975195 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.975199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.976793 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.976808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.978262 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.978268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.978586 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.978613 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.978751 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.978912 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":25}
I20260812 16:51:44.979187 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.972067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:52617 (local address 127.8.195.129:33859)
0812 16:51:44.972295 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.972300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.972483 (+ 183us) server_negotiation.cc:407] Connection header received
0812 16:51:44.972717 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.972723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.972818 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.972934 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:44.973481 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.975008 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.976974 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.978090 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.978277 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.978354 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.978776 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.978910 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:51:44.980448 11617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5421beeb8994376b9e09f6a44a4163e" candidate_uuid: "74a2008db63741cc8d6b0316208d4624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256238c37f174e899b11baf28176ceef" is_pre_election: true
I20260812 16:51:44.981081 11617 raft_consensus.cc:2454] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74a2008db63741cc8d6b0316208d4624 in term 0.
I20260812 16:51:44.981874 12043 leader_election.cc:302] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256238c37f174e899b11baf28176ceef, 74a2008db63741cc8d6b0316208d4624; no voters:
I20260812 16:51:44.982311 12329 raft_consensus.cc:2781] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:44.982517 12329 raft_consensus.cc:482] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:44.982671 12329 raft_consensus.cc:3037] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:44.984581 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.972966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:52743 (local address 127.8.195.130:40761)
0812 16:51:44.973177 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:44.973210 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:44.977884 (+ 4674us) server_negotiation.cc:407] Connection header received
0812 16:51:44.978095 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:44.978102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:44.978173 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:44.978283 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:44.979091 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.980237 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.982403 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.983495 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.983640 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.983696 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:44.984222 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 16:51:44.984344 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:51:44.984869 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:44.972512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40761 (local address 127.8.195.131:52743)
0812 16:51:44.977750 (+ 5238us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:44.977768 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:44.977948 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:44.978303 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:44.978307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:44.978335 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:44.978911 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.978921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.980390 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.980395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.982238 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:44.982254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:44.983814 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:44.983823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:44.984006 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:44.984054 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 16:51:44.984162 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:51:44.984312 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5062,"mutex_wait_us":1,"thread_start_us":4877,"threads_started":1}
I20260812 16:51:44.986066 12329 raft_consensus.cc:504] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.986056 11894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5421beeb8994376b9e09f6a44a4163e" candidate_uuid: "74a2008db63741cc8d6b0316208d4624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da465ab6092746bba4d10c2500b1e7ca" is_pre_election: true
I20260812 16:51:44.986677 11894 raft_consensus.cc:2454] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74a2008db63741cc8d6b0316208d4624 in term 0.
I20260812 16:51:44.986938 12329 leader_election.cc:288] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [CANDIDATE]: Term 1 election: Requested vote from peers 256238c37f174e899b11baf28176ceef (127.8.195.129:33859), da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761)
I20260812 16:51:44.987473 11617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5421beeb8994376b9e09f6a44a4163e" candidate_uuid: "74a2008db63741cc8d6b0316208d4624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256238c37f174e899b11baf28176ceef"
I20260812 16:51:44.987722 11617 raft_consensus.cc:3037] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:44.987759 11894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5421beeb8994376b9e09f6a44a4163e" candidate_uuid: "74a2008db63741cc8d6b0316208d4624" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da465ab6092746bba4d10c2500b1e7ca"
I20260812 16:51:44.987939 11894 raft_consensus.cc:3037] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:44.991432 11617 raft_consensus.cc:2454] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74a2008db63741cc8d6b0316208d4624 in term 1.
I20260812 16:51:44.991456 11894 raft_consensus.cc:2454] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74a2008db63741cc8d6b0316208d4624 in term 1.
I20260812 16:51:44.991858 12043 leader_election.cc:302] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74a2008db63741cc8d6b0316208d4624, da465ab6092746bba4d10c2500b1e7ca; no voters:
I20260812 16:51:44.992164 12329 raft_consensus.cc:2781] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:44.992748 12329 raft_consensus.cc:686] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 1 LEADER]: Becoming Leader. State: Replica: 74a2008db63741cc8d6b0316208d4624, State: Running, Role: LEADER
I20260812 16:51:44.993429 12329 consensus_queue.cc:237] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } }
I20260812 16:51:44.998167 11284 catalog_manager.cc:5495] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74a2008db63741cc8d6b0316208d4624 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "74a2008db63741cc8d6b0316208d4624" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } health_report { overall_health: UNKNOWN } } }
W20260812 16:51:45.027606 12278 tablet.cc:2369] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:51:45.046653 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.040599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.8.195.131:42611)
0812 16:51:45.040835 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.040839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.040862 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:51:45.041212 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.041224 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.041304 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.041432 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:45.041825 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.042670 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.044227 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.044740 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.044839 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.045499 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.046094 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.046215 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.046369 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.046439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:51:45.060075 11894 raft_consensus.cc:1261] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Refusing update from remote peer 74a2008db63741cc8d6b0316208d4624: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:45.060081 11617 raft_consensus.cc:1261] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Refusing update from remote peer 74a2008db63741cc8d6b0316208d4624: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:45.061405 12334 consensus_queue.cc:1035] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 16:51:45.063213 12329 consensus_queue.cc:1035] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:45.087828 12338 mvcc.cc:205] Tried to move back new op lower bound from 7317723156701515776 to 7317723156457168896. Current Snapshot: MvccSnapshot[applied={T|T < 7317723156701515776}]
I20260812 16:51:45.088764 12342 mvcc.cc:205] Tried to move back new op lower bound from 7317723156701515776 to 7317723156457168896. Current Snapshot: MvccSnapshot[applied={T|T < 7317723156701515776}]
I20260812 16:51:45.106467 11284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:45.106765 11284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:45.111076 11284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Sending DeleteTablet for 3 replicas of tablet e5421beeb8994376b9e09f6a44a4163e
I20260812 16:51:45.112555 12150 tablet_service.cc:1514] Processing DeleteTablet for tablet e5421beeb8994376b9e09f6a44a4163e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:48478
I20260812 16:51:45.112591 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet e5421beeb8994376b9e09f6a44a4163e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:34510
I20260812 16:51:45.112866 11561 tablet_service.cc:1514] Processing DeleteTablet for tablet e5421beeb8994376b9e09f6a44a4163e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:49438
I20260812 16:51:45.119341 11284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:45.121187 11284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:45.123240 12368 tablet_replica.cc:330] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: stopping tablet replica
I20260812 16:51:45.124027 12368 raft_consensus.cc:2229] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:45.124713 12368 raft_consensus.cc:2258] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.126772 12369 tablet_replica.cc:330] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: stopping tablet replica
I20260812 16:51:45.127207 12366 tablet_replica.cc:330] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: stopping tablet replica
I20260812 16:51:45.127265 12369 raft_consensus.cc:2229] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:45.127595 12368 ts_tablet_manager.cc:1905] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.127709 12366 raft_consensus.cc:2229] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:45.128527 12366 raft_consensus.cc:2258] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.129909 12369 raft_consensus.cc:2258] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.131129 12366 ts_tablet_manager.cc:1905] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.132091 11561 tablet_service.cc:1467] Processing CreateTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 (DEFAULT_TABLE table=test-table [id=d737c846838b4b22b925efab8099ab3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:45.132604 11561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98eb2fc2ad074cb2a23e57c847a232b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.135872 12368 ts_tablet_manager.cc:1918] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:51:45.136209 12368 log.cc:1196] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/e5421beeb8994376b9e09f6a44a4163e
I20260812 16:51:45.137312 12368 ts_tablet_manager.cc:1939] T e5421beeb8994376b9e09f6a44a4163e P 256238c37f174e899b11baf28176ceef: Deleting consensus metadata
I20260812 16:51:45.137674 12145 tablet_service.cc:1467] Processing CreateTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 (DEFAULT_TABLE table=test-table [id=d737c846838b4b22b925efab8099ab3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:45.138238 12145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98eb2fc2ad074cb2a23e57c847a232b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.138690 12369 ts_tablet_manager.cc:1905] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.139917 11265 catalog_manager.cc:4841] TS 256238c37f174e899b11baf28176ceef (127.8.195.129:33859): tablet e5421beeb8994376b9e09f6a44a4163e (table test-table [id=a0c13aab5415409494563d445a70715f]) successfully deleted
I20260812 16:51:45.141923 11853 tablet_service.cc:1467] Processing CreateTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 (DEFAULT_TABLE table=test-table [id=d737c846838b4b22b925efab8099ab3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:45.142411 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98eb2fc2ad074cb2a23e57c847a232b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.145758 12366 ts_tablet_manager.cc:1918] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:51:45.145917 12366 log.cc:1196] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/e5421beeb8994376b9e09f6a44a4163e
I20260812 16:51:45.146482 12366 ts_tablet_manager.cc:1939] T e5421beeb8994376b9e09f6a44a4163e P 74a2008db63741cc8d6b0316208d4624: Deleting consensus metadata
I20260812 16:51:45.148312 11260 catalog_manager.cc:4841] TS 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611): tablet e5421beeb8994376b9e09f6a44a4163e (table test-table [id=a0c13aab5415409494563d445a70715f]) successfully deleted
I20260812 16:51:45.148587 12369 ts_tablet_manager.cc:1918] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:51:45.148813 12369 log.cc:1196] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/e5421beeb8994376b9e09f6a44a4163e
I20260812 16:51:45.149511 12369 ts_tablet_manager.cc:1939] T e5421beeb8994376b9e09f6a44a4163e P da465ab6092746bba4d10c2500b1e7ca: Deleting consensus metadata
I20260812 16:51:45.151201 12317 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Bootstrap starting.
I20260812 16:51:45.152153 12318 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Bootstrap starting.
I20260812 16:51:45.152931 12317 tablet_bootstrap.cc:654] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.154152 12318 tablet_bootstrap.cc:654] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.155362 12316 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Bootstrap starting.
I20260812 16:51:45.155428 11265 catalog_manager.cc:4841] TS da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761): tablet e5421beeb8994376b9e09f6a44a4163e (table test-table [id=a0c13aab5415409494563d445a70715f]) successfully deleted
I20260812 16:51:45.158283 12316 tablet_bootstrap.cc:654] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.167755 12318 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: No bootstrap required, opened a new log
I20260812 16:51:45.167935 12316 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: No bootstrap required, opened a new log
I20260812 16:51:45.168005 12318 ts_tablet_manager.cc:1397] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260812 16:51:45.168098 12316 ts_tablet_manager.cc:1397] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260812 16:51:45.168908 12318 raft_consensus.cc:348] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.168946 12316 raft_consensus.cc:348] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.169224 12316 raft_consensus.cc:374] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.169207 12318 raft_consensus.cc:374] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.169327 12316 raft_consensus.cc:729] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da465ab6092746bba4d10c2500b1e7ca, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.169375 12318 raft_consensus.cc:729] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256238c37f174e899b11baf28176ceef, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.169698 12316 consensus_queue.cc:260] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.169720 12318 consensus_queue.cc:260] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.170447 12316 ts_tablet_manager.cc:1428] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:51:45.171131 12318 ts_tablet_manager.cc:1428] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:45.171764 12317 tablet_bootstrap.cc:492] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: No bootstrap required, opened a new log
I20260812 16:51:45.171943 12317 ts_tablet_manager.cc:1397] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.009s
I20260812 16:51:45.172982 12317 raft_consensus.cc:348] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.173236 12317 raft_consensus.cc:374] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.173367 12317 raft_consensus.cc:729] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a2008db63741cc8d6b0316208d4624, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.173729 12317 consensus_queue.cc:260] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.175072 12317 ts_tablet_manager.cc:1428] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 16:51:45.339774 12327 raft_consensus.cc:482] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:45.340041 12327 raft_consensus.cc:504] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.341871 12327 leader_election.cc:288] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761), 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611)
I20260812 16:51:45.351886 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.342561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:53271 (local address 127.8.195.130:40761)
0812 16:51:45.342806 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.342816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.343931 (+ 1115us) server_negotiation.cc:407] Connection header received
0812 16:51:45.344056 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.344062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.344144 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.344244 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:45.345392 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.346549 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.349500 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.350290 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.350510 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.350628 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.351187 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.351329 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:51:45.352116 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.342549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:57337 (local address 127.8.195.131:42611)
0812 16:51:45.342743 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.342750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.343671 (+ 921us) server_negotiation.cc:407] Connection header received
0812 16:51:45.344011 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.344017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.344117 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.344178 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:45.345589 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.346383 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.349452 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.350898 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.351068 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.351134 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.351629 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.351779 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:51:45.352622 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.342179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:42611 (local address 127.8.195.129:57337)
0812 16:51:45.343541 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.343565 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.343824 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.344487 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.344491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.344548 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:45.345437 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.345448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.346535 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.346540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.349283 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.349299 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.351183 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.351190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.351354 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.351423 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.351583 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.351720 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":188,"spinlock_wait_cycles":1792,"thread_start_us":1368,"threads_started":1}
I20260812 16:51:45.352644 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.342449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40761 (local address 127.8.195.129:53271)
0812 16:51:45.343820 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.343836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.343943 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.344307 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.344312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.344337 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:45.345083 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.345114 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.346725 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.346731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.349284 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.349299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.350684 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.350713 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.350942 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.351004 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.352117 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.352253 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"mutex_wait_us":33,"spinlock_wait_cycles":14464,"thread_start_us":757,"threads_started":1}
I20260812 16:51:45.353550 11892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98eb2fc2ad074cb2a23e57c847a232b0" candidate_uuid: "256238c37f174e899b11baf28176ceef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da465ab6092746bba4d10c2500b1e7ca" is_pre_election: true
I20260812 16:51:45.353761 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98eb2fc2ad074cb2a23e57c847a232b0" candidate_uuid: "256238c37f174e899b11baf28176ceef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a2008db63741cc8d6b0316208d4624" is_pre_election: true
I20260812 16:51:45.354036 11892 raft_consensus.cc:2454] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256238c37f174e899b11baf28176ceef in term 0.
I20260812 16:51:45.354173 12200 raft_consensus.cc:2454] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256238c37f174e899b11baf28176ceef in term 0.
I20260812 16:51:45.354696 11479 leader_election.cc:302] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256238c37f174e899b11baf28176ceef, da465ab6092746bba4d10c2500b1e7ca; no voters:
I20260812 16:51:45.355221 12327 raft_consensus.cc:2781] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:45.355363 12327 raft_consensus.cc:482] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:45.355475 12327 raft_consensus.cc:3037] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.357977 12327 raft_consensus.cc:504] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.358708 12327 leader_election.cc:288] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [CANDIDATE]: Term 1 election: Requested vote from peers da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761), 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611)
I20260812 16:51:45.359275 11892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98eb2fc2ad074cb2a23e57c847a232b0" candidate_uuid: "256238c37f174e899b11baf28176ceef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da465ab6092746bba4d10c2500b1e7ca"
I20260812 16:51:45.359483 11892 raft_consensus.cc:3037] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.361944 11892 raft_consensus.cc:2454] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256238c37f174e899b11baf28176ceef in term 1.
I20260812 16:51:45.362231 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98eb2fc2ad074cb2a23e57c847a232b0" candidate_uuid: "256238c37f174e899b11baf28176ceef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a2008db63741cc8d6b0316208d4624"
I20260812 16:51:45.362462 11479 leader_election.cc:302] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256238c37f174e899b11baf28176ceef, da465ab6092746bba4d10c2500b1e7ca; no voters:
I20260812 16:51:45.362488 12200 raft_consensus.cc:3037] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.362905 12327 raft_consensus.cc:2781] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:45.365082 12200 raft_consensus.cc:2454] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256238c37f174e899b11baf28176ceef in term 1.
I20260812 16:51:45.365825 12327 raft_consensus.cc:686] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 1 LEADER]: Becoming Leader. State: Replica: 256238c37f174e899b11baf28176ceef, State: Running, Role: LEADER
I20260812 16:51:45.366427 12327 consensus_queue.cc:237] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.371169 11284 catalog_manager.cc:5495] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef reported cstate change: term changed from 0 to 1, leader changed from <none> to 256238c37f174e899b11baf28176ceef (127.8.195.129). New cstate: current_term: 1 leader_uuid: "256238c37f174e899b11baf28176ceef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } health_report { overall_health: UNKNOWN } } }
Aug 12 16:51:45 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:45.478202 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.459858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37715 (local address 127.0.0.1:42772)
0812 16:51:45.460663 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.460688 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.460883 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.461366 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.461372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.462341 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:45.462616 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.462630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.463949 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.463955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.465865 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.465883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.466757 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.466765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.466964 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.467748 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:45.467781 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:45.471811 (+ 4030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:45.475268 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:45.475283 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:45.475292 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:45.475851 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:45.476247 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:45.476253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:45.476256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:45.476396 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:45.476929 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:45.476938 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:45.477147 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.477398 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.477715 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":425,"threads_started":1}
I20260812 16:51:45.478264 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.459928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.8.195.190:37715)
0812 16:51:45.461005 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.461010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.461031 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:51:45.461076 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.461080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.461133 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.461299 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:45.462855 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.463782 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.466049 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.466537 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.466649 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.472048 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:45.472054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:45.472057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:45.472086 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:45.475087 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:45.475978 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:45.475983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:45.475988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:45.476125 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:45.476507 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:45.476513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:45.476517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:45.476795 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:45.476954 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.477488 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.477860 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":978,"thread_start_us":850,"threads_started":1}
I20260812 16:51:45.484031 11284 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:42772:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:51:45.485378 11284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:45.492870 11561 tablet_service.cc:1467] Processing CreateTablet for tablet ffc4677ecd2349a595004b0f59063f4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e06b82aeb4a43648998a639dce017d6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:45.492973 11853 tablet_service.cc:1467] Processing CreateTablet for tablet ffc4677ecd2349a595004b0f59063f4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e06b82aeb4a43648998a639dce017d6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:45.493558 11561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc4677ecd2349a595004b0f59063f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.493687 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc4677ecd2349a595004b0f59063f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.499219 12316 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: Bootstrap starting.
I20260812 16:51:45.500105 12150 tablet_service.cc:1467] Processing CreateTablet for tablet ffc4677ecd2349a595004b0f59063f4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e06b82aeb4a43648998a639dce017d6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:45.500698 12150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc4677ecd2349a595004b0f59063f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:45.501125 12316 tablet_bootstrap.cc:654] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.502089 12318 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: Bootstrap starting.
I20260812 16:51:45.504086 12318 tablet_bootstrap.cc:654] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.513358 12317 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: Bootstrap starting.
I20260812 16:51:45.515992 12317 tablet_bootstrap.cc:654] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:45.517459 12318 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: No bootstrap required, opened a new log
I20260812 16:51:45.517707 12318 ts_tablet_manager.cc:1397] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260812 16:51:45.518561 12318 raft_consensus.cc:348] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.518920 12318 raft_consensus.cc:374] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.519063 12318 raft_consensus.cc:729] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256238c37f174e899b11baf28176ceef, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.520794 12318 consensus_queue.cc:260] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.521479 12327 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.521941 12327 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.526708 12317 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: No bootstrap required, opened a new log
I20260812 16:51:45.527017 12317 ts_tablet_manager.cc:1397] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260812 16:51:45.527623 12318 ts_tablet_manager.cc:1428] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260812 16:51:45.528376 12317 raft_consensus.cc:348] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.528745 12317 raft_consensus.cc:374] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.528836 12317 raft_consensus.cc:729] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a2008db63741cc8d6b0316208d4624, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.530167 12317 consensus_queue.cc:260] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.530670 12334 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.530998 12316 tablet_bootstrap.cc:492] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: No bootstrap required, opened a new log
I20260812 16:51:45.531021 12334 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.531195 12316 ts_tablet_manager.cc:1397] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.020s
I20260812 16:51:45.532038 12317 ts_tablet_manager.cc:1428] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260812 16:51:45.532019 12316 raft_consensus.cc:348] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.532275 12316 raft_consensus.cc:374] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:45.532390 12316 raft_consensus.cc:729] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da465ab6092746bba4d10c2500b1e7ca, State: Initialized, Role: FOLLOWER
I20260812 16:51:45.532718 12316 consensus_queue.cc:260] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.533771 12316 ts_tablet_manager.cc:1428] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:51:45.533713 12330 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.534147 12330 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.680706 12330 raft_consensus.cc:482] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:45.680953 12330 raft_consensus.cc:504] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.682736 12330 leader_election.cc:288] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 256238c37f174e899b11baf28176ceef (127.8.195.129:33859), 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611)
I20260812 16:51:45.702040 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.683536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:59307 (local address 127.8.195.131:42611)
0812 16:51:45.683760 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.683766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.693855 (+ 10089us) server_negotiation.cc:407] Connection header received
0812 16:51:45.694238 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.694247 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.694352 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.694415 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:45.695247 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.696297 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.699203 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.700430 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.700597 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.700674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.701336 (+ 662us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.701499 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:51:45.702091 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.683009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:42611 (local address 127.8.195.130:59307)
0812 16:51:45.693658 (+ 10649us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.693688 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.694027 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.694611 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.694615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.694651 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:45.695048 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.695063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.696448 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.696454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.699000 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.699018 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.700829 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.700837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.700986 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.701016 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.701307 (+ 291us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.701480 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10457,"mutex_wait_us":172,"thread_start_us":10689,"threads_started":1}
I20260812 16:51:45.703366 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc4677ecd2349a595004b0f59063f4a" candidate_uuid: "da465ab6092746bba4d10c2500b1e7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a2008db63741cc8d6b0316208d4624" is_pre_election: true
I20260812 16:51:45.703717 12200 raft_consensus.cc:2454] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da465ab6092746bba4d10c2500b1e7ca in term 0.
I20260812 16:51:45.704334 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.683192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:42655 (local address 127.8.195.129:33859)
0812 16:51:45.694560 (+ 11368us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.694566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.694586 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:51:45.694653 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.694658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.694732 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.694830 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:45.695478 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.696258 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.700773 (+ 4515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.702953 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.703240 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.703313 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.703735 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.703967 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11245,"thread_start_us":11140,"threads_started":1}
I20260812 16:51:45.704337 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.683009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33859 (local address 127.8.195.130:42655)
0812 16:51:45.693970 (+ 10961us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.693995 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.694120 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.694870 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.694876 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.694907 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:45.695315 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.695325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.696404 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.696413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.699004 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.699018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.703310 (+ 4292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.703322 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.703529 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.703571 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.703683 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.703851 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10823,"mutex_wait_us":331,"thread_start_us":10144,"threads_started":1}
I20260812 16:51:45.704371 11741 leader_election.cc:302] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74a2008db63741cc8d6b0316208d4624, da465ab6092746bba4d10c2500b1e7ca; no voters:
I20260812 16:51:45.704936 12330 raft_consensus.cc:2781] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:45.705084 12330 raft_consensus.cc:482] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:45.705302 12330 raft_consensus.cc:3037] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.705440 11617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc4677ecd2349a595004b0f59063f4a" candidate_uuid: "da465ab6092746bba4d10c2500b1e7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256238c37f174e899b11baf28176ceef" is_pre_election: true
I20260812 16:51:45.705814 11617 raft_consensus.cc:2454] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da465ab6092746bba4d10c2500b1e7ca in term 0.
I20260812 16:51:45.708441 12330 raft_consensus.cc:504] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.709110 12330 leader_election.cc:288] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [CANDIDATE]: Term 1 election: Requested vote from peers 256238c37f174e899b11baf28176ceef (127.8.195.129:33859), 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611)
I20260812 16:51:45.709625 11617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc4677ecd2349a595004b0f59063f4a" candidate_uuid: "da465ab6092746bba4d10c2500b1e7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256238c37f174e899b11baf28176ceef"
I20260812 16:51:45.709928 11617 raft_consensus.cc:3037] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.709884 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc4677ecd2349a595004b0f59063f4a" candidate_uuid: "da465ab6092746bba4d10c2500b1e7ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a2008db63741cc8d6b0316208d4624"
I20260812 16:51:45.710232 12200 raft_consensus.cc:3037] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:45.714049 11617 raft_consensus.cc:2454] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da465ab6092746bba4d10c2500b1e7ca in term 1.
I20260812 16:51:45.714071 12200 raft_consensus.cc:2454] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da465ab6092746bba4d10c2500b1e7ca in term 1.
I20260812 16:51:45.714596 11741 leader_election.cc:302] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 256238c37f174e899b11baf28176ceef, 74a2008db63741cc8d6b0316208d4624, da465ab6092746bba4d10c2500b1e7ca; no voters:
I20260812 16:51:45.715035 12330 raft_consensus.cc:2781] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:45.715584 12330 raft_consensus.cc:686] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [term 1 LEADER]: Becoming Leader. State: Replica: da465ab6092746bba4d10c2500b1e7ca, State: Running, Role: LEADER
I20260812 16:51:45.716054 12330 consensus_queue.cc:237] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } }
I20260812 16:51:45.718164 12418 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: TxnStatusTablet state changed. Reason: New leader da465ab6092746bba4d10c2500b1e7ca. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.719031 12418 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:45.719975 12424 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:51:45.720273 12424 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:51:45.720342 11282 catalog_manager.cc:5495] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca reported cstate change: term changed from 0 to 1, leader changed from <none> to da465ab6092746bba4d10c2500b1e7ca (127.8.195.130). New cstate: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } health_report { overall_health: UNKNOWN } } }
I20260812 16:51:45.763275 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.755516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40761 (local address 127.0.0.1:34518)
0812 16:51:45.755940 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.755955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.756114 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.756793 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.756797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.756808 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:45.756967 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.756974 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.758968 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.758972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.760044 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.760057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.761147 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.761151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.761387 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.761449 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.762716 (+ 1267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.762735 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.762818 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.762943 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260812 16:51:45.763342 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.755496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.8.195.130:40761)
0812 16:51:45.755709 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.755714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.756046 (+ 332us) server_negotiation.cc:407] Connection header received
0812 16:51:45.756356 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.756366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.756577 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.756732 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:45.757116 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.758838 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.760219 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.760987 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.761137 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.761656 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.762611 (+ 955us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.762763 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.762842 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.762932 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:51:45.769512 11617 raft_consensus.cc:1261] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Refusing update from remote peer da465ab6092746bba4d10c2500b1e7ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:45.769558 12200 raft_consensus.cc:1261] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Refusing update from remote peer da465ab6092746bba4d10c2500b1e7ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:45.770406 12418 consensus_queue.cc:1035] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:45.770752 12330 consensus_queue.cc:1035] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:45.788444 12334 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: TxnStatusTablet state changed. Reason: New leader da465ab6092746bba4d10c2500b1e7ca. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.788826 12334 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.792219 12327 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: TxnStatusTablet state changed. Reason: New leader da465ab6092746bba4d10c2500b1e7ca. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.792778 12327 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.792881 12330 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.793190 12330 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:45.796005 12418 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.796808 12418 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P da465ab6092746bba4d10c2500b1e7ca: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:45.798403 12327 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.798733 12327 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 256238c37f174e899b11baf28176ceef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.802533 12334 tablet_replica.cc:439] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da465ab6092746bba4d10c2500b1e7ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256238c37f174e899b11baf28176ceef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33859 } } peers { permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 } } peers { permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 } } }
I20260812 16:51:45.802935 12334 tablet_replica.cc:442] T ffc4677ecd2349a595004b0f59063f4a P 74a2008db63741cc8d6b0316208d4624: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:45.818387 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.807745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.8.195.129:33859)
0812 16:51:45.807938 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.807945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.809803 (+ 1858us) server_negotiation.cc:407] Connection header received
0812 16:51:45.810143 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.810149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.810336 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.810499 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:45.811428 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.812730 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.815300 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.816102 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.816226 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.816827 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.817636 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.817790 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.817932 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.818060 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:51:45.838658 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.831425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40761 (local address 127.8.195.129:46257)
0812 16:51:45.832219 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.832235 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.832355 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.832717 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.832720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.832729 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:45.833035 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.833043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.834750 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.834754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.836183 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.836197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.837383 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.837388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.837589 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.837659 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.838140 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.838148 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.838212 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.838310 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":467,"threads_started":1}
I20260812 16:51:45.838704 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.831329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:46257 (local address 127.8.195.130:40761)
0812 16:51:45.831578 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.831583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.832312 (+ 729us) server_negotiation.cc:407] Connection header received
0812 16:51:45.832491 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.832498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.832590 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.832701 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:45.833215 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.834596 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.836322 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.837209 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.837341 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.837886 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.838049 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.838143 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.838277 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.838359 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:51:45.844889 12327 consensus_queue.cc:1035] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a2008db63741cc8d6b0316208d4624" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:45.856632 12385 consensus_queue.cc:1035] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [LEADER]: Connected to new peer: Peer: permanent_uuid: "da465ab6092746bba4d10c2500b1e7ca" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:45.899322 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.889402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:40931 (local address 127.8.195.129:33859)
0812 16:51:45.889620 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.889628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.890696 (+ 1068us) server_negotiation.cc:407] Connection header received
0812 16:51:45.890936 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.890941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.891047 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.891141 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:45.892064 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.893351 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.896075 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.896591 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.896737 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.898323 (+ 1586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.898513 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.898622 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:45.898746 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 16:51:45.898825 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:51:45.899387 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.889373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33859 (local address 127.8.195.130:40931)
0812 16:51:45.890583 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:45.890604 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:45.890770 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:45.891311 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:45.891319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:45.891335 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:45.891853 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.891869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.893475 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.893479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.895905 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:45.895924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:45.896808 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:45.896816 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:45.897009 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:45.897072 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:45.898626 (+ 1554us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:45.898641 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:51:45.898716 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:51:45.898869 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"thread_start_us":641,"threads_started":1}
I20260812 16:51:45.941250 11282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:45.941735 11282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:45.945562 11282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 023c1c68868f423ebcabf91e4a0605b7: Sending DeleteTablet for 3 replicas of tablet 98eb2fc2ad074cb2a23e57c847a232b0
I20260812 16:51:45.946556 12150 tablet_service.cc:1514] Processing DeleteTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:48478
I20260812 16:51:45.946718 11561 tablet_service.cc:1514] Processing DeleteTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:49438
I20260812 16:51:45.947574 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet 98eb2fc2ad074cb2a23e57c847a232b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:45 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:34510
I20260812 16:51:45.959403 12477 tablet_replica.cc:330] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: stopping tablet replica
I20260812 16:51:45.959646 12477 raft_consensus.cc:2229] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:45.959971 12477 raft_consensus.cc:2258] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.961143 12477 ts_tablet_manager.cc:1905] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.975126 12477 ts_tablet_manager.cc:1918] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:45.975314 12477 log.cc:1196] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/98eb2fc2ad074cb2a23e57c847a232b0
I20260812 16:51:45.975822 12477 ts_tablet_manager.cc:1939] T 98eb2fc2ad074cb2a23e57c847a232b0 P 74a2008db63741cc8d6b0316208d4624: Deleting consensus metadata
I20260812 16:51:45.977404 11260 catalog_manager.cc:4841] TS 74a2008db63741cc8d6b0316208d4624 (127.8.195.131:42611): tablet 98eb2fc2ad074cb2a23e57c847a232b0 (table test-table [id=d737c846838b4b22b925efab8099ab3f]) successfully deleted
I20260812 16:51:45.980580 12479 tablet_replica.cc:330] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: stopping tablet replica
I20260812 16:51:45.980835 12479 raft_consensus.cc:2229] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:45.981354 12479 raft_consensus.cc:2258] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.982826 12479 ts_tablet_manager.cc:1905] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.983476 12478 tablet_replica.cc:330] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: stopping tablet replica
I20260812 16:51:45.984982 12478 raft_consensus.cc:2229] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:45.985772 12478 raft_consensus.cc:2258] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:45.987167 12478 ts_tablet_manager.cc:1905] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:45.992400 12479 ts_tablet_manager.cc:1918] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:45.992559 12479 log.cc:1196] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/98eb2fc2ad074cb2a23e57c847a232b0
I20260812 16:51:45.993062 12479 ts_tablet_manager.cc:1939] T 98eb2fc2ad074cb2a23e57c847a232b0 P da465ab6092746bba4d10c2500b1e7ca: Deleting consensus metadata
I20260812 16:51:45.994324 11265 catalog_manager.cc:4841] TS da465ab6092746bba4d10c2500b1e7ca (127.8.195.130:40761): tablet 98eb2fc2ad074cb2a23e57c847a232b0 (table test-table [id=d737c846838b4b22b925efab8099ab3f]) successfully deleted
I20260812 16:51:45.998620 12478 ts_tablet_manager.cc:1918] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:45.999255 12478 log.cc:1196] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/98eb2fc2ad074cb2a23e57c847a232b0
I20260812 16:51:46.000070 12478 ts_tablet_manager.cc:1939] T 98eb2fc2ad074cb2a23e57c847a232b0 P 256238c37f174e899b11baf28176ceef: Deleting consensus metadata
I20260812 16:51:46.001660 11265 catalog_manager.cc:4841] TS 256238c37f174e899b11baf28176ceef (127.8.195.129:33859): tablet 98eb2fc2ad074cb2a23e57c847a232b0 (table test-table [id=d737c846838b4b22b925efab8099ab3f]) successfully deleted
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:46.020296 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:45.984150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.8.195.129:33859)
0812 16:51:45.984368 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:45.984375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:45.995341 (+ 10966us) server_negotiation.cc:407] Connection header received
0812 16:51:45.995701 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:45.995710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:45.995796 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:45.995875 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:45.999338 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.000784 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.003753 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.004576 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.004754 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:46.010767 (+ 6013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:46.010784 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:46.010805 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:46.010854 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:46.016185 (+ 5331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.017386 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.017397 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.017400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.017574 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.018071 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.018077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.018081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.018684 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:46.019016 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:46.019625 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 16:51:46.019884 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 16:51:46.022163 11690 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49442
I20260812 16:51:46.066026 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:46.037059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.8.195.190:37715)
0812 16:51:46.047639 (+ 10580us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:46.047647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:46.048249 (+ 602us) server_negotiation.cc:407] Connection header received
0812 16:51:46.048510 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:46.048515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:46.048604 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:46.048751 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:46.050842 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.052379 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.054328 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.054979 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.055115 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:46.058423 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:46.058431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:46.058436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:46.058478 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:46.062504 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.063498 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.063504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.063507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.063716 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.064165 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.064170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.064174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.064530 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:46.064813 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:46.065230 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 16:51:46.065488 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10386,"thread_start_us":9928,"threads_started":1}
W20260812 16:51:46.067214 11282 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42778
I20260812 16:51:46.069700 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11445
I20260812 16:51:46.091751 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:46.071643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.8.195.129:33859)
0812 16:51:46.071877 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:46.071886 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:46.072396 (+ 510us) server_negotiation.cc:407] Connection header received
0812 16:51:46.072567 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:46.072573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:46.072668 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:46.072780 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:46.074531 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.075880 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.078128 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.079024 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.079225 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:46.082861 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:46.082869 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:46.082874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:46.082918 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:46.087660 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.088744 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.088751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.088755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.088957 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.089608 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.089614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.089618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.090054 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:46.090393 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:46.090894 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 16:51:46.091240 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260812 16:51:46.092545 11690 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49458
W20260812 16:51:46.093283 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:46.093560 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11445
I20260812 16:51:46.127305 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11704
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:51:46.145359 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:46.128134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.8.195.130:40761)
0812 16:51:46.128380 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:46.128387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:46.128411 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:51:46.128573 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:46.128579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:46.128669 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:46.128804 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:46.130131 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.131332 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.132647 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.133479 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.133627 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:46.137973 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:46.137986 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:46.138002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:46.138052 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:46.142336 (+ 4284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.143082 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.143092 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.143096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.143278 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.143649 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.143654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.143657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.144154 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:46.144520 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:46.144667 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 16:51:46.144903 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260812 16:51:46.146246 11972 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34528
W20260812 16:51:46.146979 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11704: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:46.147168 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11704
I20260812 16:51:46.172827 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11992
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[11201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553504, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:51:46.188637 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:46.173712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.8.195.131:42611)
0812 16:51:46.173898 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:46.173902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:46.173921 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:51:46.173981 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:46.173984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:46.174062 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:46.174155 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:46.175474 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.176254 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.177600 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:46.178034 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:46.178137 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:46.182490 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:46.182507 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:46.182523 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:46.182564 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:46.185725 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.186442 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.186451 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.186453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.186599 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:46.187004 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:46.187010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:46.187013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:46.187388 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:46.187744 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:46.188021 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 16:51:46.188279 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 16:51:46.189401 12265 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48496
W20260812 16:51:46.190032 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11992: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:46.190191 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11992
I20260812 16:51:46.220746 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11227
I20260812 16:51:46.221940 11384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:46.338094 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11227
2026-08-12T16:51:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5361 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[12502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[12502](info): set up 2 sockets
Aug 12 16:51:46 dist-test-slave-z8x9 krb5kdc[12502](info): commencing operation
krb5kdc: starting...
W20260812 16:51:47.730281 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:51:47 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:51:47.964425 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36215
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:36215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:48.297429 12524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:48.297802 12524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:48.297971 12524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:48.305374 12524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:51:48.305536 12524 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:51:48.305634 12524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:48.305766 12524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:48.305856 12524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:51:48.305948 12524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:51:48.315857 12524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:36215
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:48.317252 12524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:48.318596 12524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:48.330444 12545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:48.330483 12542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:48.330521 12541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:48.331879 12524 server_base.cc:1034] running on GCE node
I20260812 16:51:48.332613 12524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:48.334008 12524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:48.335269 12524 hybrid_clock.cc:648] HybridClock initialized: now 1786553508335191 us; error 40 us; skew 500 ppm
I20260812 16:51:48.335577 12524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:48 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:48.338972 12524 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:51:48.340494 12524 webserver.cc:460] Webserver started at http://127.8.195.190:32981/ using document root <none> and password file <none>
I20260812 16:51:48.341248 12524 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:48.341359 12524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:48.341707 12524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:48.344766 12524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "82a0e617e481460985eba58df12c97a4"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "433674dfdfc545ec750e227977307106"
server_key_iv: "6b8e35288eedb76a92023179a523637b"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.345587 12524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "82a0e617e481460985eba58df12c97a4"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "433674dfdfc545ec750e227977307106"
server_key_iv: "6b8e35288eedb76a92023179a523637b"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.350927 12524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:48.354395 12562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:48.355733 12524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:48.355927 12524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "82a0e617e481460985eba58df12c97a4"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "433674dfdfc545ec750e227977307106"
server_key_iv: "6b8e35288eedb76a92023179a523637b"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.356184 12524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:48.375017 12524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:48.379181 12524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:48.379470 12524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:48.401870 12524 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:36215
I20260812 16:51:48.401908 12711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:36215 every 8 connection(s)
I20260812 16:51:48.403251 12524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:51:48.407137 12712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:48.410171 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 12524
I20260812 16:51:48.410413 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:51:48.411067 8974 external_mini_cluster.cc:1428] Setting key 691c5ef5f5ef6fc65f2408535d1a5b2c
I20260812 16:51:48.420452 12712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Bootstrap starting.
I20260812 16:51:48.424230 12712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:48.425284 12712 log.cc:824] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:51:48 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:48.428421 12712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: No bootstrap required, opened a new log
I20260812 16:51:48.434988 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:48.413096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.8.195.190:36215)
0812 16:51:48.414019 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:48.414035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:48.414084 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:51:48.416145 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:48.416181 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:48.416714 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:48.417209 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:48.418598 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:48.420583 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:48.421891 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:48.422982 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:48.423178 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:48.427545 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:48.427554 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:48.427568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:48.427607 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:48.430743 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:48.431571 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:48.431577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:48.431583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:48.431761 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:48.432165 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:48.432168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:48.432170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:48.432690 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:48.432916 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:48.433476 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 16:51:48.433912 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":364,"threads_started":1}
I20260812 16:51:48.437358 12712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } }
I20260812 16:51:48.437736 12712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:48.437867 12712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82a0e617e481460985eba58df12c97a4, State: Initialized, Role: FOLLOWER
I20260812 16:51:48.438426 12712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } }
I20260812 16:51:48.438663 12712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:51:48.438824 12712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:51:48.438995 12712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:48.441118 12712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } }
I20260812 16:51:48.441699 12712 leader_election.cc:302] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82a0e617e481460985eba58df12c97a4; no voters:
I20260812 16:51:48.442380 12712 leader_election.cc:288] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:51:48.442483 12721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:48.443171 12721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [term 1 LEADER]: Becoming Leader. State: Replica: 82a0e617e481460985eba58df12c97a4, State: Running, Role: LEADER
I20260812 16:51:48.443710 12721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } }
I20260812 16:51:48.444626 12712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:51:48.446933 12722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82a0e617e481460985eba58df12c97a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } } }
I20260812 16:51:48.447295 12722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:48.447979 12724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82a0e617e481460985eba58df12c97a4. Latest consensus state: current_term: 1 leader_uuid: "82a0e617e481460985eba58df12c97a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a0e617e481460985eba58df12c97a4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36215 } } }
I20260812 16:51:48.448274 12724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:48.449548 12733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:51:48.454751 12733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:51:48.463910 12733 catalog_manager.cc:1347] Generated new cluster ID: 5276d4b9c15f425e8c67737cb74e8c6c
I20260812 16:51:48.464042 12733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:51:48.484990 12733 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:51:48.486393 12733 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:51:48.503273 12733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Generated new TSK 0
I20260812 16:51:48.504421 12733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:51:48.570389 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:36215
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:48.904867 12757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:48.905227 12757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:48.905413 12757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:48.912606 12757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:48.912757 12757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:48.912919 12757 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:51:48.913009 12757 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:48.913152 12757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:51:48.922976 12757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:48.924341 12757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:48.925909 12757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:48.939432 12773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:48.939550 12770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:48.939585 12767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:48.940645 12757 server_base.cc:1034] running on GCE node
I20260812 16:51:48.941399 12757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:48.942343 12757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:48.943578 12757 hybrid_clock.cc:648] HybridClock initialized: now 1786553508943540 us; error 52 us; skew 500 ppm
I20260812 16:51:48.943888 12757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:48 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:48.952375 12757 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:51:48.954036 12757 webserver.cc:460] Webserver started at http://127.8.195.129:39861/ using document root <none> and password file <none>
I20260812 16:51:48.954702 12757 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:48.954809 12757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:48.955082 12757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:48.958132 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "e39827546333968554f4f839c98d80b6"
server_key_iv: "920560675a1bb3a1d0abc1fbbd04acac"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.959004 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "e39827546333968554f4f839c98d80b6"
server_key_iv: "920560675a1bb3a1d0abc1fbbd04acac"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.964380 12757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:51:48.967955 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:48.969499 12757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:48.969705 12757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
format_stamp: "Formatted at 2026-08-12 16:51:48 on dist-test-slave-z8x9"
server_key: "e39827546333968554f4f839c98d80b6"
server_key_iv: "920560675a1bb3a1d0abc1fbbd04acac"
server_key_version: "encryptionkey@0"
I20260812 16:51:48.969918 12757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:49.002447 12757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:49.007876 12757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:49.008213 12757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:49.011332 12757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:49.011487 12757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:49.011615 12757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:49.011698 12757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:49 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:49.035485 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.011714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49579 (local address 127.8.195.190:36215)
0812 16:51:49.012187 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:49.012194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:49.013481 (+ 1287us) server_negotiation.cc:407] Connection header received
0812 16:51:49.015552 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:49.015558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:49.015657 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:49.015775 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:49.018262 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.019575 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.021261 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.022037 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.022192 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.027687 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:49.027695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:49.027701 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:49.027776 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:49.030974 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.032184 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.032193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.032198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.032375 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.033103 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.033111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.033118 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.033608 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:49.033795 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:49.034587 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 16:51:49.034933 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":260,"threads_started":1}
I20260812 16:51:49.036098 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.012454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.129:49579)
0812 16:51:49.013325 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:49.013369 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:49.015245 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:49.016047 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:49.016066 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:49.016976 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:49.017981 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.018004 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.019786 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.019790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.021072 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.021085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.022362 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.022373 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.022755 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.023694 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:49.023722 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:49.027387 (+ 3665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:49.031134 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.031148 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.031176 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.032004 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.032532 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.032535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.032537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.032948 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.033798 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:49.033806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:49.034164 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 16:51:49.034396 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 16:51:49.034808 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":341,"threads_started":1}
I20260812 16:51:49.060938 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.043329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:57265 (local address 127.8.195.190:36215)
0812 16:51:49.043578 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:49.043587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:49.044138 (+ 551us) server_negotiation.cc:407] Connection header received
0812 16:51:49.044343 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:49.044347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:49.044417 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:49.044497 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:49.046285 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.047528 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.050405 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.051121 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.051255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.053794 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:49.053803 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:49.053809 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:49.053850 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:49.057562 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.058561 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.058567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.058571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.058757 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.059301 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.059307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.059310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.059641 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:49.059783 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:49.060398 (+ 615us) server_negotiation.cc:299] Negotiation successful
0812 16:51:49.060623 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:51:49.061517 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.043254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.129:57265)
0812 16:51:49.044011 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:49.044037 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:49.044213 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:49.044569 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:49.044577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:49.045453 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:49.046063 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.046085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.047771 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.047783 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.050210 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.050235 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.051586 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.051596 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.051826 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.052546 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:49.052568 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:49.053513 (+ 945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:49.057729 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.057733 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.057735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.058424 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.058904 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.058921 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.058931 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.059138 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.059787 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:49.059796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:49.060069 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 16:51:49.060250 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 16:51:49.060752 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":345,"threads_started":1}
I20260812 16:51:49.067477 12757 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:35415
I20260812 16:51:49.067517 13049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:35415 every 8 connection(s)
I20260812 16:51:49.068734 12757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:51:49.070829 13050 heartbeater.cc:347] Connected to a master server at 127.8.195.190:36215
I20260812 16:51:49.071071 13050 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:49.071694 13050 heartbeater.cc:511] Master 127.8.195.190:36215 requested a full tablet report, sending...
I20260812 16:51:49.073900 12598 ts_manager.cc:194] Registered new tserver with Master: cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415)
I20260812 16:51:49.074091 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 12757
I20260812 16:51:49.074313 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:51:49.075132 8974 external_mini_cluster.cc:1428] Setting key c9b20d7e4919bcaf7eded213e3a7aa9c
I20260812 16:51:49.075917 12598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:49579
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:51:49.122771 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:36215
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:49.451313 13054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:49.451612 13054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:49.451771 13054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:49.458922 13054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:49.459100 13054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:49.459245 13054 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:51:49.459334 13054 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:49.459414 13054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:51:49.469246 13054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:49.470480 13054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:49.471853 13054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:49.485430 13061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:49.485478 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:49.485522 13062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:49.486572 13054 server_base.cc:1034] running on GCE node
I20260812 16:51:49.487190 13054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:49.488073 13054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:49.489284 13054 hybrid_clock.cc:648] HybridClock initialized: now 1786553509489239 us; error 32 us; skew 500 ppm
I20260812 16:51:49.489552 13054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:49 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:49.492702 13054 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:51:49.494151 13054 webserver.cc:460] Webserver started at http://127.8.195.130:39751/ using document root <none> and password file <none>
I20260812 16:51:49.494781 13054 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:49.494884 13054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:49.495146 13054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:49.497978 13054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "17248d0a4a2943a79a312c8f350a61f2"
format_stamp: "Formatted at 2026-08-12 16:51:49 on dist-test-slave-z8x9"
server_key: "96a6ff6bf913df763ecf074b0ad658bc"
server_key_iv: "14af581aa2c7f5e44bd3bf73ef408738"
server_key_version: "encryptionkey@0"
I20260812 16:51:49.498679 13054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "17248d0a4a2943a79a312c8f350a61f2"
format_stamp: "Formatted at 2026-08-12 16:51:49 on dist-test-slave-z8x9"
server_key: "96a6ff6bf913df763ecf074b0ad658bc"
server_key_iv: "14af581aa2c7f5e44bd3bf73ef408738"
server_key_version: "encryptionkey@0"
I20260812 16:51:49.503669 13054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:51:49.507170 13082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:49.508450 13054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:49.508620 13054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "17248d0a4a2943a79a312c8f350a61f2"
format_stamp: "Formatted at 2026-08-12 16:51:49 on dist-test-slave-z8x9"
server_key: "96a6ff6bf913df763ecf074b0ad658bc"
server_key_iv: "14af581aa2c7f5e44bd3bf73ef408738"
server_key_version: "encryptionkey@0"
I20260812 16:51:49.508832 13054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:49.530988 13054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:49.535570 13054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:49.535883 13054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:49.538782 13054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:49.538930 13054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:49.539055 13054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:49.539142 13054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:49 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553509, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:49.562914 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.539708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:41591 (local address 127.8.195.190:36215)
0812 16:51:49.539926 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:49.539937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:49.541520 (+ 1583us) server_negotiation.cc:407] Connection header received
0812 16:51:49.543830 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:49.543840 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:49.543946 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:49.544085 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:49.546819 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.547761 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.549828 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.550521 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.550684 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.555906 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:49.555915 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:49.555924 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:49.555960 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:49.559538 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.560367 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.560374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.560377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.560516 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.561014 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.561021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.561025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.561403 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:49.561575 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:49.562307 (+ 732us) server_negotiation.cc:299] Negotiation successful
0812 16:51:49.562534 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:51:49.563987 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.540417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.130:41591)
0812 16:51:49.541346 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:49.541395 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:49.543427 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:49.544460 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:49.544471 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:49.545413 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:49.546532 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.546571 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.547927 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.547933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.549578 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.549601 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.550827 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.550835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.551215 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.552047 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:49.552070 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:49.555650 (+ 3580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:49.559696 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.559705 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.559720 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.560232 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.560651 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.560656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.560660 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.560864 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.561546 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:49.561554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:49.561919 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 16:51:49.562206 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 16:51:49.562751 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":361,"threads_started":1}
I20260812 16:51:49.585916 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.569938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:60823 (local address 127.8.195.190:36215)
0812 16:51:49.570151 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:49.570158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:49.570859 (+ 701us) server_negotiation.cc:407] Connection header received
0812 16:51:49.571050 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:49.571054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:49.571112 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:49.571189 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:49.572617 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.573575 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.575680 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.576284 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.576424 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.578966 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:49.578976 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:49.578984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:49.579033 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:49.582903 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.583805 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.583813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.583816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.583961 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:49.584342 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:49.584348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:49.584353 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:49.584706 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:49.584878 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:49.585305 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 16:51:49.585551 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:51:49.586047 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:49.569950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.130:60823)
0812 16:51:49.570737 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:49.570761 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:49.570929 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:49.571251 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:49.571259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:49.571977 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:49.572463 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.572480 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.573727 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.573732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.575485 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:49.575507 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:49.576494 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:49.576503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:49.576683 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:49.577472 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:49.577503 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:49.578718 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:49.583039 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.583044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.583049 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.583675 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.584077 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:49.584082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:49.584086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:49.584222 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:49.584838 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:49.584843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:49.585035 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 16:51:49.585217 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 16:51:49.585597 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":399,"threads_started":1}
I20260812 16:51:49.599066 13054 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:41623
I20260812 16:51:49.599084 13355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:41623 every 8 connection(s)
I20260812 16:51:49.600466 13054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:51:49.602574 13357 heartbeater.cc:347] Connected to a master server at 127.8.195.190:36215
I20260812 16:51:49.602785 13357 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:49.603367 13357 heartbeater.cc:511] Master 127.8.195.190:36215 requested a full tablet report, sending...
I20260812 16:51:49.604230 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13054
I20260812 16:51:49.604436 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:51:49.604791 12595 ts_manager.cc:194] Registered new tserver with Master: 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:49.605014 8974 external_mini_cluster.cc:1428] Setting key bc8cd541d339f55c14e52d6120fc7296
I20260812 16:51:49.605984 12595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:41591
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:51:49.654518 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:36215
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:49.992722 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:49.993080 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:49.993306 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:50.000815 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:50.000988 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:50.001159 13365 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 16:51:50.001295 13365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:51:50.001415 13365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:51:50.011808 13365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:36215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:50.013334 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:50.014779 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:50.028244 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:50.028407 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:50.028434 13374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:50.029604 13365 server_base.cc:1034] running on GCE node
I20260812 16:51:50.030330 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:50.031245 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:50.032460 13365 hybrid_clock.cc:648] HybridClock initialized: now 1786553510032421 us; error 35 us; skew 500 ppm
I20260812 16:51:50.032806 13365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:50 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553510, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:50.036046 13365 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:51:50.037602 13365 webserver.cc:460] Webserver started at http://127.8.195.131:34073/ using document root <none> and password file <none>
I20260812 16:51:50.038323 13365 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:50.038471 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:50.038816 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:50.041791 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "36356a8bdf9141f79c1ddce89bbfac95"
format_stamp: "Formatted at 2026-08-12 16:51:50 on dist-test-slave-z8x9"
server_key: "f7e352ac8455d6216665def12bff8ecd"
server_key_iv: "58281f486009fab589f51b081d92dff1"
server_key_version: "encryptionkey@0"
I20260812 16:51:50.042557 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "36356a8bdf9141f79c1ddce89bbfac95"
format_stamp: "Formatted at 2026-08-12 16:51:50 on dist-test-slave-z8x9"
server_key: "f7e352ac8455d6216665def12bff8ecd"
server_key_iv: "58281f486009fab589f51b081d92dff1"
server_key_version: "encryptionkey@0"
I20260812 16:51:50.048120 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:51:50.051842 13392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:50.053129 13365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:50.053411 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "36356a8bdf9141f79c1ddce89bbfac95"
format_stamp: "Formatted at 2026-08-12 16:51:50 on dist-test-slave-z8x9"
server_key: "f7e352ac8455d6216665def12bff8ecd"
server_key_iv: "58281f486009fab589f51b081d92dff1"
server_key_version: "encryptionkey@0"
I20260812 16:51:50.053627 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:50.072870 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:50.077682 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:50.077984 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:50.078905 13050 heartbeater.cc:503] Master 127.8.195.190:36215 was elected leader, sending a full tablet report...
I20260812 16:51:50.081348 13365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:50.081477 13365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:50.081619 13365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:50.081717 13365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:51:50 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553510, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:50.106612 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.081780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:36969 (local address 127.8.195.190:36215)
0812 16:51:50.081973 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.082009 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.083438 (+ 1429us) server_negotiation.cc:407] Connection header received
0812 16:51:50.085626 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.085634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.085738 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.085862 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:50.088384 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.089530 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.092343 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.092766 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.092900 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.097923 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:50.097929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:50.097933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:50.097968 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:50.102142 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.103472 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.103482 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.103488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.103677 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.104319 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.104326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.104332 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.104822 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:50.104994 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.106004 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.106237 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:51:50.107764 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.082488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.131:36969)
0812 16:51:50.083290 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.083329 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.085271 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.086251 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.086263 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.087063 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:50.088125 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.088157 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.089720 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.089724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.092176 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.092189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.093025 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.093033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.093322 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.094110 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:50.094133 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:50.097650 (+ 3517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:50.102368 (+ 4718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.102383 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.102407 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.103302 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.103847 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.103854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.103859 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.104170 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.104978 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:50.104984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:50.105547 (+ 563us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.105887 (+ 340us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.106476 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":352,"threads_started":1}
I20260812 16:51:50.130128 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.114449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:54873 (local address 127.8.195.190:36215)
0812 16:51:50.114611 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.114615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.115307 (+ 692us) server_negotiation.cc:407] Connection header received
0812 16:51:50.115504 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.115510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.115592 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.115695 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:50.116653 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.117456 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.119534 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.120121 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.120289 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.123113 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:50.123123 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:50.123130 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:50.123179 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:50.126715 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.127779 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.127788 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.127794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.127980 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.128479 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.128486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.128491 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.128826 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:50.129007 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.129509 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.129752 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:51:50.130244 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.114579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.8.195.131:54873)
0812 16:51:50.115190 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.115209 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.115376 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.115724 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.115728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.116198 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:50.116509 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.116518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.117612 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.117616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.119381 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.119393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.120416 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.120429 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.120682 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.121545 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:50.121563 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:50.122909 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:50.126858 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.126863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.126868 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.127542 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.128089 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.128094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.128098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.128295 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.128954 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:50.128959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:50.129220 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.129417 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.129775 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":350,"threads_started":1}
I20260812 16:51:50.137674 13365 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:45751
I20260812 16:51:50.137719 13675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:45751 every 8 connection(s)
I20260812 16:51:50.138913 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:51:50.141122 13676 heartbeater.cc:347] Connected to a master server at 127.8.195.190:36215
I20260812 16:51:50.141390 13676 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:50.142071 13676 heartbeater.cc:511] Master 127.8.195.190:36215 requested a full tablet report, sending...
I20260812 16:51:50.143417 12598 ts_manager.cc:194] Registered new tserver with Master: 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751)
I20260812 16:51:50.144205 12598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:36969
I20260812 16:51:50.147344 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13365
I20260812 16:51:50.147540 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:51:50.148017 8974 external_mini_cluster.cc:1428] Setting key ddc97886ae7ffc0b4c4ff4db01d5a4e7
I20260812 16:51:50.151278 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:51:50 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:51:50 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:50.209283 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.194583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.8.195.190:36215)
0812 16:51:50.194820 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.194826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.194994 (+ 168us) server_negotiation.cc:407] Connection header received
0812 16:51:50.195155 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.195160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.195224 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.195333 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:50.197121 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.198034 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.199327 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.199727 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.199821 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.204126 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:50.204132 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:50.204135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:50.204169 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:50.206759 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.207373 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.207377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.207379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.207498 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.207961 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.207966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.207968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.208270 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:50.208453 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.208734 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.208902 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:51:50.213872 12598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:50.220786 12598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:50.255761 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.234889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.8.195.130:41623)
0812 16:51:50.247029 (+ 12140us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.247042 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.247077 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:51:50.247204 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.247213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.247543 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.247806 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.248479 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.250025 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.253050 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.254541 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.254682 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.254907 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.255282 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.255454 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11940,"thread_start_us":12584,"threads_started":1}
I20260812 16:51:50.255954 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.234497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.0.0.1:35900)
0812 16:51:50.246863 (+ 12366us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.246884 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.247001 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.247812 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.247820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.247850 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.248275 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.248288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.250222 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.250227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.252875 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.252890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.254872 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.254879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.255043 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.255079 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.255228 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.255419 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12231,"mutex_wait_us":36,"thread_start_us":12590,"threads_started":1}
I20260812 16:51:50.257409 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.233461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.8.195.131:45751)
0812 16:51:50.248995 (+ 15534us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.249005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.249050 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:51:50.249144 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.249152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.249438 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.250323 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.250678 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.252040 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.254499 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.256217 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.256351 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.256594 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.256841 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.257026 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15359,"thread_start_us":15170,"threads_started":1}
I20260812 16:51:50.257403 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.233202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45751 (local address 127.0.0.1:33636)
0812 16:51:50.234067 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.234107 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.234297 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.249762 (+ 15465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.249767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.249797 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.250276 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.250288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.252185 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.252190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.254293 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.254310 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.256441 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.256451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.256639 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.256670 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.256786 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.256937 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":22,"thread_start_us":659,"threads_started":1}
I20260812 16:51:50.257074 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.233346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:35415 (local address 127.0.0.1:53154)
0812 16:51:50.245662 (+ 12316us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.245691 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.245883 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.249062 (+ 3179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.249069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.249097 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.249633 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.249650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.251580 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.251586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.253764 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.253781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.256164 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.256171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.256379 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.256414 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.256484 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.256626 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12104,"mutex_wait_us":25,"thread_start_us":12601,"threads_started":1}
I20260812 16:51:50.258596 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.233660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.8.195.129:35415)
0812 16:51:50.248473 (+ 14813us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.248482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.248510 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:51:50.248608 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.248617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.248841 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.249042 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.249861 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.251423 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.253974 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.255918 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.257844 (+ 1926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.258044 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.258111 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.258287 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14624,"thread_start_us":14463,"threads_started":1}
I20260812 16:51:50.260433 13186 tablet_service.cc:1467] Processing CreateTablet for tablet c50702d9ed6746658966e6decb936c1f (DEFAULT_TABLE table=test-table [id=ac7407274c7d4218b2d6c3e086b19517]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.260880 13512 tablet_service.cc:1467] Processing CreateTablet for tablet c50702d9ed6746658966e6decb936c1f (DEFAULT_TABLE table=test-table [id=ac7407274c7d4218b2d6c3e086b19517]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.262055 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50702d9ed6746658966e6decb936c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.262149 13512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50702d9ed6746658966e6decb936c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.262149 12897 tablet_service.cc:1467] Processing CreateTablet for tablet c50702d9ed6746658966e6decb936c1f (DEFAULT_TABLE table=test-table [id=ac7407274c7d4218b2d6c3e086b19517]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.263538 12897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50702d9ed6746658966e6decb936c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.274031 13717 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Bootstrap starting.
I20260812 16:51:50.276726 13719 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Bootstrap starting.
I20260812 16:51:50.278327 13720 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Bootstrap starting.
I20260812 16:51:50.278993 13717 tablet_bootstrap.cc:654] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.280319 13719 tablet_bootstrap.cc:654] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.280421 13717 log.cc:824] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:50.281126 13720 tablet_bootstrap.cc:654] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.281682 13719 log.cc:824] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:50.282399 13720 log.cc:824] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:50.283736 13717 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: No bootstrap required, opened a new log
I20260812 16:51:50.284036 13717 ts_tablet_manager.cc:1397] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 16:51:50.285000 13720 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: No bootstrap required, opened a new log
I20260812 16:51:50.285300 13720 ts_tablet_manager.cc:1397] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260812 16:51:50.286428 13719 tablet_bootstrap.cc:492] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: No bootstrap required, opened a new log
I20260812 16:51:50.286688 13719 ts_tablet_manager.cc:1397] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260812 16:51:50.292065 13717 raft_consensus.cc:348] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.292435 13717 raft_consensus.cc:374] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.292570 13717 raft_consensus.cc:729] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.293234 13717 consensus_queue.cc:260] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.294389 13676 heartbeater.cc:503] Master 127.8.195.190:36215 was elected leader, sending a full tablet report...
I20260812 16:51:50.294919 13717 ts_tablet_manager.cc:1428] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260812 16:51:50.296685 13719 raft_consensus.cc:348] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.297060 13719 raft_consensus.cc:374] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.296921 13720 raft_consensus.cc:348] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.297235 13719 raft_consensus.cc:729] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.297407 13720 raft_consensus.cc:374] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.297516 13720 raft_consensus.cc:729] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.297945 13719 consensus_queue.cc:260] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.298100 13720 consensus_queue.cc:260] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.299073 13357 heartbeater.cc:503] Master 127.8.195.190:36215 was elected leader, sending a full tablet report...
I20260812 16:51:50.299753 13719 ts_tablet_manager.cc:1428] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 16:51:50.302982 13720 ts_tablet_manager.cc:1428] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
I20260812 16:51:50.303817 13736 raft_consensus.cc:482] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:50.304113 13736 raft_consensus.cc:504] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.305895 13736 leader_election.cc:288] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:50.315831 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.306208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36661 (local address 127.8.195.130:41623)
0812 16:51:50.306414 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.306424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.309039 (+ 2615us) server_negotiation.cc:407] Connection header received
0812 16:51:50.309281 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.309287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.309377 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.309482 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.310542 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.311668 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.314122 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.314898 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.315083 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.315160 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.315439 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.315564 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:51:50.316155 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.306185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.8.195.129:36661)
0812 16:51:50.308903 (+ 2718us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.308929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.309108 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.309516 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.309526 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.309551 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.310354 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.310379 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.311810 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.311815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.313954 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.313967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.315118 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.315124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.315275 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.315299 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.315403 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.315536 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2543,"mutex_wait_us":24,"thread_start_us":2327,"threads_started":1}
I20260812 16:51:50.316313 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.309904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:52209 (local address 127.8.195.131:45751)
0812 16:51:50.310112 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.310121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.310737 (+ 616us) server_negotiation.cc:407] Connection header received
0812 16:51:50.310880 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.310884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.310940 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.311007 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.311591 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.312394 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.314588 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.315366 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.315551 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.315629 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.315929 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.316056 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 16:51:50.316468 13744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.309934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45751 (local address 127.8.195.129:52209)
0812 16:51:50.310613 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.310651 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.310767 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.311051 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.311055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.311079 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.311443 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.311454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.312536 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.312541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.314425 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.314438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.315598 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.315605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.315767 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.315793 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.315907 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.316027 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":3,"thread_start_us":356,"threads_started":1}
I20260812 16:51:50.317472 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50702d9ed6746658966e6decb936c1f" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2" is_pre_election: true
I20260812 16:51:50.317473 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50702d9ed6746658966e6decb936c1f" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95" is_pre_election: true
I20260812 16:51:50.318051 13548 raft_consensus.cc:2454] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 0.
I20260812 16:51:50.318051 13238 raft_consensus.cc:2454] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 0.
I20260812 16:51:50.318753 12792 leader_election.cc:302] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95, cb3042f47e3e4a98b9a5f50c42a5c2f2; no voters:
I20260812 16:51:50.319190 13737 raft_consensus.cc:2781] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:50.319350 13737 raft_consensus.cc:482] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:50.319525 13737 raft_consensus.cc:3037] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Advancing to term 1
W20260812 16:51:50.321215 13051 tablet.cc:2369] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:51:50.322275 13737 raft_consensus.cc:504] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.322826 13737 leader_election.cc:288] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 election: Requested vote from peers 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:50.323305 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50702d9ed6746658966e6decb936c1f" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95"
I20260812 16:51:50.323457 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50702d9ed6746658966e6decb936c1f" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2"
I20260812 16:51:50.323532 13548 raft_consensus.cc:3037] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.323671 13238 raft_consensus.cc:3037] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.326946 13238 raft_consensus.cc:2454] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 1.
I20260812 16:51:50.326946 13548 raft_consensus.cc:2454] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 1.
I20260812 16:51:50.327412 12802 leader_election.cc:302] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36356a8bdf9141f79c1ddce89bbfac95, cb3042f47e3e4a98b9a5f50c42a5c2f2; no voters:
I20260812 16:51:50.327704 13737 raft_consensus.cc:2781] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:50.327903 13737 raft_consensus.cc:686] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 LEADER]: Becoming Leader. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Running, Role: LEADER
I20260812 16:51:50.328444 13737 consensus_queue.cc:237] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } }
I20260812 16:51:50.333645 12598 catalog_manager.cc:5495] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129). New cstate: current_term: 1 leader_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } health_report { overall_health: UNKNOWN } } }
W20260812 16:51:50.352517 13358 tablet.cc:2369] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:51:50 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
W20260812 16:51:50.390578 13677 tablet.cc:2369] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:51:50.393349 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.379492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.8.195.190:36215)
0812 16:51:50.379705 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.379714 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.380495 (+ 781us) server_negotiation.cc:407] Connection header received
0812 16:51:50.380704 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.380710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.380785 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.380867 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:50.382162 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.382915 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.384161 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.384599 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.384693 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.387998 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:50.388003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:50.388006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:50.388034 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:50.390565 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.391247 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.391254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.391269 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.391384 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:50.391774 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:50.391782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:50.391787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:50.392063 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:50.392212 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.392623 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.392978 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":75}
I20260812 16:51:50.393386 13759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.379544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36215 (local address 127.0.0.1:51370)
0812 16:51:50.380357 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.380382 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.380560 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.380919 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.380922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.381745 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:50.382020 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.382029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.383038 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.383042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.384028 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.384040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.384772 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.384777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.384896 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.385632 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:50.385654 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:50.387824 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:50.390687 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.390702 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.390711 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.391128 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.391507 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:50.391511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:50.391515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:50.391663 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:50.392192 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:50.392199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:50.392373 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.392558 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.392863 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":3,"thread_start_us":432,"threads_started":1}
I20260812 16:51:50.398245 12598 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:51370:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:51:50.399164 12598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:50.406239 13512 tablet_service.cc:1467] Processing CreateTablet for tablet b7bc0409f8794b04b2ad4ab8cfc19843 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7088f2722e1e4099b76f6339c02cca94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:50.406116 12897 tablet_service.cc:1467] Processing CreateTablet for tablet b7bc0409f8794b04b2ad4ab8cfc19843 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7088f2722e1e4099b76f6339c02cca94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:50.407064 13512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bc0409f8794b04b2ad4ab8cfc19843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.407068 12897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bc0409f8794b04b2ad4ab8cfc19843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.409406 13186 tablet_service.cc:1467] Processing CreateTablet for tablet b7bc0409f8794b04b2ad4ab8cfc19843 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7088f2722e1e4099b76f6339c02cca94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:51:50.410323 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7bc0409f8794b04b2ad4ab8cfc19843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.413244 13717 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: Bootstrap starting.
I20260812 16:51:50.414588 13717 tablet_bootstrap.cc:654] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.417387 13719 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: Bootstrap starting.
I20260812 16:51:50.417515 13717 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: No bootstrap required, opened a new log
I20260812 16:51:50.417644 13717 ts_tablet_manager.cc:1397] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 16:51:50.418432 13717 raft_consensus.cc:348] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.418700 13717 raft_consensus.cc:374] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.418803 13717 raft_consensus.cc:729] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.419021 13719 tablet_bootstrap.cc:654] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.419027 13717 consensus_queue.cc:260] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.419672 13717 ts_tablet_manager.cc:1428] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:51:50.419498 13729 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.419924 13729 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.422992 13719 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: No bootstrap required, opened a new log
I20260812 16:51:50.423139 13719 ts_tablet_manager.cc:1397] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 16:51:50.423231 13720 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Bootstrap starting.
I20260812 16:51:50.423898 13719 raft_consensus.cc:348] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.424136 13719 raft_consensus.cc:374] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.424223 13719 raft_consensus.cc:729] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.424490 13719 consensus_queue.cc:260] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.424986 13720 tablet_bootstrap.cc:654] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.424927 13733 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.425314 13733 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.426321 13719 ts_tablet_manager.cc:1428] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:50.428290 13720 tablet_bootstrap.cc:492] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: No bootstrap required, opened a new log
I20260812 16:51:50.428444 13720 ts_tablet_manager.cc:1397] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 16:51:50.429112 13720 raft_consensus.cc:348] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.429402 13720 raft_consensus.cc:374] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.429514 13720 raft_consensus.cc:729] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.429741 13720 consensus_queue.cc:260] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.430150 13737 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.430346 13720 ts_tablet_manager.cc:1428] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:51:50.430449 13737 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.488953 13733 raft_consensus.cc:482] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:50.489250 13733 raft_consensus.cc:504] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.490979 13733 leader_election.cc:288] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751)
I20260812 16:51:50.500203 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.491327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:35153 (local address 127.8.195.129:35415)
0812 16:51:50.491617 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.491627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.492134 (+ 507us) server_negotiation.cc:407] Connection header received
0812 16:51:50.492458 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.492462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.492567 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.492674 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.493602 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.494732 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.497813 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.499066 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.499243 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.499307 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.499808 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.499944 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:51:50.500223 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.491981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:60961 (local address 127.8.195.131:45751)
0812 16:51:50.492180 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.492191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.492391 (+ 200us) server_negotiation.cc:407] Connection header received
0812 16:51:50.492549 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.492554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.492632 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.492732 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.493559 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.494753 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.497815 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.499079 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.499229 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.499305 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.499808 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.499944 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 16:51:50.500447 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.491214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:35415 (local address 127.8.195.130:35153)
0812 16:51:50.492010 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.492037 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.492272 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.492728 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.492733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.492819 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.493435 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.493448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.494884 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.494889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.497632 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.497647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.499371 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.499376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.499595 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.499664 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.499755 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.499920 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":277,"spinlock_wait_cycles":44416,"thread_start_us":381,"threads_started":1}
I20260812 16:51:50.500785 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.491270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45751 (local address 127.8.195.130:60961)
0812 16:51:50.492291 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.492307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.492416 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.492742 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.492746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.492789 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.493332 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.493350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.494883 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.494897 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.497632 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.497647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.499352 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.499364 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.499596 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.499655 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.499755 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.499916 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":180,"thread_start_us":719,"threads_started":1}
I20260812 16:51:50.501441 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bc0409f8794b04b2ad4ab8cfc19843" candidate_uuid: "17248d0a4a2943a79a312c8f350a61f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" is_pre_election: true
I20260812 16:51:50.501610 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bc0409f8794b04b2ad4ab8cfc19843" candidate_uuid: "17248d0a4a2943a79a312c8f350a61f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95" is_pre_election: true
I20260812 16:51:50.501883 13548 raft_consensus.cc:2454] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17248d0a4a2943a79a312c8f350a61f2 in term 0.
I20260812 16:51:50.501895 12944 raft_consensus.cc:2454] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17248d0a4a2943a79a312c8f350a61f2 in term 0.
I20260812 16:51:50.502415 13088 leader_election.cc:302] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95; no voters:
I20260812 16:51:50.502857 13733 raft_consensus.cc:2781] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:50.502970 13733 raft_consensus.cc:482] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:50.503104 13733 raft_consensus.cc:3037] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.505360 13733 raft_consensus.cc:504] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.505975 13733 leader_election.cc:288] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [CANDIDATE]: Term 1 election: Requested vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751)
I20260812 16:51:50.506413 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bc0409f8794b04b2ad4ab8cfc19843" candidate_uuid: "17248d0a4a2943a79a312c8f350a61f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
I20260812 16:51:50.506448 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7bc0409f8794b04b2ad4ab8cfc19843" candidate_uuid: "17248d0a4a2943a79a312c8f350a61f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95"
I20260812 16:51:50.506616 12944 raft_consensus.cc:3037] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.506659 13548 raft_consensus.cc:3037] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.508786 13548 raft_consensus.cc:2454] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17248d0a4a2943a79a312c8f350a61f2 in term 1.
I20260812 16:51:50.508982 12944 raft_consensus.cc:2454] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17248d0a4a2943a79a312c8f350a61f2 in term 1.
I20260812 16:51:50.509238 13088 leader_election.cc:302] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95; no voters:
I20260812 16:51:50.509526 13733 raft_consensus.cc:2781] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:50.509970 13733 raft_consensus.cc:686] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 LEADER]: Becoming Leader. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Running, Role: LEADER
I20260812 16:51:50.510443 13733 consensus_queue.cc:237] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.514057 13778 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: TxnStatusTablet state changed. Reason: New leader 17248d0a4a2943a79a312c8f350a61f2. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.514397 13778 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:50.515154 13780 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:51:50.515374 13780 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:51:50.516271 12595 catalog_manager.cc:5495] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130). New cstate: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } health_report { overall_health: UNKNOWN } } }
I20260812 16:51:50.557076 13759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.550269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.0.0.1:35908)
0812 16:51:50.550600 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.550615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.550736 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.551258 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.551266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.551282 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:50.551475 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.551483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.552977 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.552981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.554098 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.554111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.555208 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.555214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.555382 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.555440 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.556553 (+ 1113us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.556576 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.556645 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.556780 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260812 16:51:50.557119 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.550203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.8.195.130:41623)
0812 16:51:50.550410 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.550417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.550707 (+ 290us) server_negotiation.cc:407] Connection header received
0812 16:51:50.550938 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.550944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.551054 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.551219 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:50.551618 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.552840 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.554302 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.555039 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.555189 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.555635 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.556442 (+ 807us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.556590 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.556677 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.556775 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:51:50.564929 13548 raft_consensus.cc:1261] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Refusing update from remote peer 17248d0a4a2943a79a312c8f350a61f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:50.564945 12944 raft_consensus.cc:1261] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Refusing update from remote peer 17248d0a4a2943a79a312c8f350a61f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:50.566022 13778 consensus_queue.cc:1035] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:50.566916 13733 consensus_queue.cc:1035] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:50.581122 13729 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: TxnStatusTablet state changed. Reason: New leader 17248d0a4a2943a79a312c8f350a61f2. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.581460 13729 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.584100 13737 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: TxnStatusTablet state changed. Reason: New leader 17248d0a4a2943a79a312c8f350a61f2. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.584400 13737 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.585870 13786 mvcc.cc:205] Tried to move back new op lower bound from 7317723179264303104 to 7317723179054559232. Current Snapshot: MvccSnapshot[applied={T|T < 7317723179264303104}]
I20260812 16:51:50.586973 13787 mvcc.cc:205] Tried to move back new op lower bound from 7317723179264303104 to 7317723179054559232. Current Snapshot: MvccSnapshot[applied={T|T < 7317723179264303104}]
I20260812 16:51:50.588428 13778 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.588719 13778 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:50.589653 13790 mvcc.cc:205] Tried to move back new op lower bound from 7317723179264303104 to 7317723179054559232. Current Snapshot: MvccSnapshot[applied={T|T < 7317723179264303104}]
I20260812 16:51:50.591615 13733 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.594135 13733 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 17248d0a4a2943a79a312c8f350a61f2: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:51:50.598263 13729 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.598600 13729 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P 36356a8bdf9141f79c1ddce89bbfac95: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.599869 13737 tablet_replica.cc:439] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17248d0a4a2943a79a312c8f350a61f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } } }
I20260812 16:51:50.600111 13737 tablet_replica.cc:442] T b7bc0409f8794b04b2ad4ab8cfc19843 P cb3042f47e3e4a98b9a5f50c42a5c2f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:51:50.614954 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.607308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.8.195.129:35415)
0812 16:51:50.607520 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.607526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.607545 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:51:50.607635 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.607641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.607740 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.607893 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:50.608367 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.609790 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.612126 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.612779 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.612907 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.613556 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.614254 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.614395 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.614532 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.614625 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:51:50.634371 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.627400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:35969 (local address 127.8.195.130:41623)
0812 16:51:50.627619 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.627626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.628617 (+ 991us) server_negotiation.cc:407] Connection header received
0812 16:51:50.628930 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.628935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.629013 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.629229 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:50.629749 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.630613 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.632647 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.633085 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.633245 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.633754 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.633852 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.633925 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.634078 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.634144 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:51:50.634614 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.627425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.8.195.129:35969)
0812 16:51:50.628496 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.628513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.628690 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.629250 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.629255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.629267 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:50.629598 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.629607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.630773 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.630779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.632495 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.632511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.633288 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.633295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.633484 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.633554 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.633955 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.633967 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.634052 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.634201 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":760,"threads_started":1}
I20260812 16:51:50.645509 13238 raft_consensus.cc:1261] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Refusing update from remote peer cb3042f47e3e4a98b9a5f50c42a5c2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:50.645561 13548 raft_consensus.cc:1261] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Refusing update from remote peer cb3042f47e3e4a98b9a5f50c42a5c2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:50.646353 13736 consensus_queue.cc:1035] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:50.646960 13737 consensus_queue.cc:1035] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:50.701431 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.689428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:49961 (local address 127.8.195.129:35415)
0812 16:51:50.689700 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.689708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.694874 (+ 5166us) server_negotiation.cc:407] Connection header received
0812 16:51:50.695119 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.695125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.695209 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.695309 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:50.696008 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.697196 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.699416 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.699905 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.700052 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.700590 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.700691 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.700808 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.700988 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.701081 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:51:50.701612 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.689356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:35415 (local address 127.8.195.130:49961)
0812 16:51:50.694778 (+ 5422us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.694801 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.694948 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.695337 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.695341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.695353 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:50.695797 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.695820 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.697374 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.697380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.699223 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.699239 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.700108 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.700115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.700313 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.700400 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:50.700810 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:50.700824 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.700951 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.701089 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5208,"thread_start_us":5045,"threads_started":1}
I20260812 16:51:50.743739 12595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:50.744069 12595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:50.748435 12595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Sending DeleteTablet for 3 replicas of tablet c50702d9ed6746658966e6decb936c1f
I20260812 16:51:50.749632 13512 tablet_service.cc:1514] Processing DeleteTablet for tablet c50702d9ed6746658966e6decb936c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:33636
I20260812 16:51:50.750474 12897 tablet_service.cc:1514] Processing DeleteTablet for tablet c50702d9ed6746658966e6decb936c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53154
I20260812 16:51:50.750579 13820 tablet_replica.cc:330] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: stopping tablet replica
I20260812 16:51:50.751024 13820 raft_consensus.cc:2229] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:50.751392 13820 raft_consensus.cc:2258] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:50.751678 13186 tablet_service.cc:1514] Processing DeleteTablet for tablet c50702d9ed6746658966e6decb936c1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35900
I20260812 16:51:50.753420 13820 ts_tablet_manager.cc:1905] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:50.754392 12595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:50.755064 12595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:50.764154 13512 tablet_service.cc:1467] Processing CreateTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b (DEFAULT_TABLE table=test-table [id=36531dad13154dc0b730ba97475faca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.764256 12893 tablet_service.cc:1467] Processing CreateTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b (DEFAULT_TABLE table=test-table [id=36531dad13154dc0b730ba97475faca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.764667 13512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14ed3fa3c445bbbd505c562c4e807b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.764870 12893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14ed3fa3c445bbbd505c562c4e807b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.770016 13183 tablet_service.cc:1467] Processing CreateTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b (DEFAULT_TABLE table=test-table [id=36531dad13154dc0b730ba97475faca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:50.770642 13183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb14ed3fa3c445bbbd505c562c4e807b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:50.772670 13720 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Bootstrap starting.
I20260812 16:51:50.774682 13720 tablet_bootstrap.cc:654] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.780700 13719 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Bootstrap starting.
I20260812 16:51:50.782344 13719 tablet_bootstrap.cc:654] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.786533 13717 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Bootstrap starting.
I20260812 16:51:50.787154 13826 tablet_replica.cc:330] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: stopping tablet replica
I20260812 16:51:50.788182 13717 tablet_bootstrap.cc:654] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:50.789004 13821 tablet_replica.cc:330] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: stopping tablet replica
I20260812 16:51:50.789628 13826 raft_consensus.cc:2229] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:50.790169 13821 raft_consensus.cc:2229] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:50.790808 13720 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: No bootstrap required, opened a new log
I20260812 16:51:50.791038 13720 ts_tablet_manager.cc:1397] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.004s
I20260812 16:51:50.791090 13826 raft_consensus.cc:2258] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:50.791682 13821 raft_consensus.cc:2258] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:50.792841 13719 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: No bootstrap required, opened a new log
I20260812 16:51:50.791889 13720 raft_consensus.cc:348] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.793018 13719 ts_tablet_manager.cc:1397] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260812 16:51:50.793134 13720 raft_consensus.cc:374] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.793299 13720 raft_consensus.cc:729] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.793614 13720 consensus_queue.cc:260] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.794461 13720 ts_tablet_manager.cc:1428] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:50.793936 13719 raft_consensus.cc:348] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.794956 13719 raft_consensus.cc:374] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.795053 13719 raft_consensus.cc:729] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.795324 13719 consensus_queue.cc:260] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.796036 13719 ts_tablet_manager.cc:1428] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:50.797940 13826 ts_tablet_manager.cc:1905] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:50.804066 13821 ts_tablet_manager.cc:1905] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:50.808419 13820 ts_tablet_manager.cc:1918] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:50.808609 13820 log.cc:1196] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/c50702d9ed6746658966e6decb936c1f
I20260812 16:51:50.809140 13820 ts_tablet_manager.cc:1939] T c50702d9ed6746658966e6decb936c1f P 36356a8bdf9141f79c1ddce89bbfac95: Deleting consensus metadata
I20260812 16:51:50.810355 13717 tablet_bootstrap.cc:492] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: No bootstrap required, opened a new log
I20260812 16:51:50.810531 13717 ts_tablet_manager.cc:1397] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.007s
I20260812 16:51:50.811367 13717 raft_consensus.cc:348] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.811609 13717 raft_consensus.cc:374] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:50.811685 13717 raft_consensus.cc:729] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Initialized, Role: FOLLOWER
I20260812 16:51:50.811949 13717 consensus_queue.cc:260] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.815169 13826 ts_tablet_manager.cc:1918] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:50.816553 13717 ts_tablet_manager.cc:1428] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260812 16:51:50.818369 13821 ts_tablet_manager.cc:1918] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:50.818630 13826 log.cc:1196] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/c50702d9ed6746658966e6decb936c1f
I20260812 16:51:50.819327 13826 ts_tablet_manager.cc:1939] T c50702d9ed6746658966e6decb936c1f P 17248d0a4a2943a79a312c8f350a61f2: Deleting consensus metadata
I20260812 16:51:50.819540 13821 log.cc:1196] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/c50702d9ed6746658966e6decb936c1f
I20260812 16:51:50.820161 13821 ts_tablet_manager.cc:1939] T c50702d9ed6746658966e6decb936c1f P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting consensus metadata
I20260812 16:51:50.820262 12570 catalog_manager.cc:4841] TS 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751): tablet c50702d9ed6746658966e6decb936c1f (table test-table [id=ac7407274c7d4218b2d6c3e086b19517]) successfully deleted
I20260812 16:51:50.822189 12567 catalog_manager.cc:4841] TS 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623): tablet c50702d9ed6746658966e6decb936c1f (table test-table [id=ac7407274c7d4218b2d6c3e086b19517]) successfully deleted
I20260812 16:51:50.823660 12568 catalog_manager.cc:4841] TS cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415): tablet c50702d9ed6746658966e6decb936c1f (table test-table [id=ac7407274c7d4218b2d6c3e086b19517]) successfully deleted
I20260812 16:51:50.935895 13729 raft_consensus.cc:482] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:50.936189 13729 raft_consensus.cc:504] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.938050 13729 leader_election.cc:288] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:50.941035 13736 raft_consensus.cc:482] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:50.941238 13736 raft_consensus.cc:504] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.941931 13736 leader_election.cc:288] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623), 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751)
I20260812 16:51:50.942816 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2" is_pre_election: true
I20260812 16:51:50.943104 13238 raft_consensus.cc:2454] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 0.
I20260812 16:51:50.943568 12792 leader_election.cc:302] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, cb3042f47e3e4a98b9a5f50c42a5c2f2; no voters:
I20260812 16:51:50.943864 13736 raft_consensus.cc:2781] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:50.943982 13736 raft_consensus.cc:482] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:50.944075 13736 raft_consensus.cc:3037] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.947269 13736 raft_consensus.cc:504] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.948333 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2"
I20260812 16:51:50.948544 13238 raft_consensus.cc:3037] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.951784 13238 raft_consensus.cc:2454] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 1.
I20260812 16:51:50.952291 12792 leader_election.cc:302] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, cb3042f47e3e4a98b9a5f50c42a5c2f2; no voters:
I20260812 16:51:50.952572 13737 raft_consensus.cc:2781] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:50.952751 13737 raft_consensus.cc:686] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 LEADER]: Becoming Leader. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Running, Role: LEADER
I20260812 16:51:50.953014 13737 consensus_queue.cc:237] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:50.959312 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.939020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:39313 (local address 127.8.195.129:35415)
0812 16:51:50.939209 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.939215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.949055 (+ 9840us) server_negotiation.cc:407] Connection header received
0812 16:51:50.949459 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.949471 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.949577 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.949649 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.950493 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.951766 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.954924 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.956263 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.956464 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.956548 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.958835 (+ 2287us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.958986 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:51:50.960434 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.938415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:35415 (local address 127.8.195.131:39313)
0812 16:51:50.948938 (+ 10523us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.948962 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.949272 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.949897 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.949901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.949939 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.950293 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.950311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.952068 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.952076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.954700 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.954719 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.958424 (+ 3705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.958430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.958622 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.958656 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.959708 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.959881 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10392,"thread_start_us":10091,"threads_started":1}
I20260812 16:51:50.961290 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" is_pre_election: true
I20260812 16:51:50.962257 13736 leader_election.cc:288] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [CANDIDATE]: Term 1 election: Requested vote from peers 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623), 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751)
I20260812 16:51:50.970316 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.939430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:33449 (local address 127.8.195.130:41623)
0812 16:51:50.939648 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:50.939666 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:50.962687 (+ 23021us) server_negotiation.cc:407] Connection header received
0812 16:51:50.962968 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:50.962976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:50.963061 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:50.963123 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:51:50.963708 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.964906 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.967580 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.969024 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.969201 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.969285 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:50.969863 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 16:51:50.970022 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:51:50.971124 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:50.938609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.8.195.131:33449)
0812 16:51:50.962591 (+ 23982us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:50.962614 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:50.962807 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:50.963336 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:50.963341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:50.963370 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:51:50.963539 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.963550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.965108 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.965114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.967365 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:50.967381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:50.969447 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:50.969454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:50.969647 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:50.969686 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 16:51:50.970718 (+ 1032us) client_negotiation.cc:240] Negotiation successful
0812 16:51:50.970877 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23870,"threads_started":1}
I20260812 16:51:50.972651 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95" is_pre_election: true
I20260812 16:51:50.972910 13548 raft_consensus.cc:2454] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 0.
I20260812 16:51:50.974391 12595 catalog_manager.cc:5495] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129). New cstate: current_term: 1 leader_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } health_report { overall_health: UNKNOWN } } }
I20260812 16:51:50.975046 13548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36356a8bdf9141f79c1ddce89bbfac95"
I20260812 16:51:50.975255 13548 raft_consensus.cc:3037] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:50.978262 13548 raft_consensus.cc:2454] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in term 1.
I20260812 16:51:50.979005 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb14ed3fa3c445bbbd505c562c4e807b" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2" is_pre_election: true
I20260812 16:51:50.979461 13238 raft_consensus.cc:2379] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 36356a8bdf9141f79c1ddce89bbfac95 in current term 1: Already voted for candidate cb3042f47e3e4a98b9a5f50c42a5c2f2 in this term.
I20260812 16:51:50.980005 13394 leader_election.cc:302] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36356a8bdf9141f79c1ddce89bbfac95; no voters: 17248d0a4a2943a79a312c8f350a61f2, cb3042f47e3e4a98b9a5f50c42a5c2f2
I20260812 16:51:50.980464 13729 raft_consensus.cc:2726] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 16:51:51.028409 13546 raft_consensus.cc:1261] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Refusing update from remote peer cb3042f47e3e4a98b9a5f50c42a5c2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:51.028460 13238 raft_consensus.cc:1261] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Refusing update from remote peer cb3042f47e3e4a98b9a5f50c42a5c2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:51.029238 13737 consensus_queue.cc:1035] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:51.029636 13736 consensus_queue.cc:1035] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:51:51.054787 13790 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:51:52.660182 12595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:52.660544 12595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:52.666808 12595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Sending DeleteTablet for 3 replicas of tablet fb14ed3fa3c445bbbd505c562c4e807b
I20260812 16:51:52.667716 12897 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53154
I20260812 16:51:52.667924 13183 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35900
I20260812 16:51:52.667917 13512 tablet_service.cc:1514] Processing DeleteTablet for tablet fb14ed3fa3c445bbbd505c562c4e807b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:33636
I20260812 16:51:52.668440 13892 tablet_replica.cc:330] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: stopping tablet replica
I20260812 16:51:52.668776 13894 tablet_replica.cc:330] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: stopping tablet replica
I20260812 16:51:52.668742 13892 raft_consensus.cc:2229] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:52.668994 13894 raft_consensus.cc:2229] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:52.669152 13892 raft_consensus.cc:2258] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:52.669315 13894 raft_consensus.cc:2258] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:52.670011 13895 tablet_replica.cc:330] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: stopping tablet replica
I20260812 16:51:52.670238 13895 raft_consensus.cc:2229] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:52.670468 13895 raft_consensus.cc:2258] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:52.671223 13894 ts_tablet_manager.cc:1905] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.673916 13895 ts_tablet_manager.cc:1905] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.677980 13892 ts_tablet_manager.cc:1905] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.677994 13894 ts_tablet_manager.cc:1918] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:52.678181 13894 log.cc:1196] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/fb14ed3fa3c445bbbd505c562c4e807b
I20260812 16:51:52.678870 13894 ts_tablet_manager.cc:1939] T fb14ed3fa3c445bbbd505c562c4e807b P 36356a8bdf9141f79c1ddce89bbfac95: Deleting consensus metadata
I20260812 16:51:52.680243 12570 catalog_manager.cc:4841] TS 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751): tablet fb14ed3fa3c445bbbd505c562c4e807b (table test-table [id=36531dad13154dc0b730ba97475faca1]) successfully deleted
I20260812 16:51:52.685302 13892 ts_tablet_manager.cc:1918] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:52.685456 13892 log.cc:1196] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/fb14ed3fa3c445bbbd505c562c4e807b
I20260812 16:51:52.685432 13895 ts_tablet_manager.cc:1918] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:52.685570 13895 log.cc:1196] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/fb14ed3fa3c445bbbd505c562c4e807b
I20260812 16:51:52.686090 13892 ts_tablet_manager.cc:1939] T fb14ed3fa3c445bbbd505c562c4e807b P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting consensus metadata
I20260812 16:51:52.686141 13895 ts_tablet_manager.cc:1939] T fb14ed3fa3c445bbbd505c562c4e807b P 17248d0a4a2943a79a312c8f350a61f2: Deleting consensus metadata
I20260812 16:51:52.687318 12568 catalog_manager.cc:4841] TS cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415): tablet fb14ed3fa3c445bbbd505c562c4e807b (table test-table [id=36531dad13154dc0b730ba97475faca1]) successfully deleted
I20260812 16:51:52.687493 12567 catalog_manager.cc:4841] TS 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623): tablet fb14ed3fa3c445bbbd505c562c4e807b (table test-table [id=36531dad13154dc0b730ba97475faca1]) successfully deleted
Aug 12 16:51:52 dist-test-slave-z8x9 krb5kdc[12502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:51:52 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:52.728506 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.713359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.8.195.190:36215)
0812 16:51:52.713970 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:52.713975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:52.714079 (+ 104us) server_negotiation.cc:407] Connection header received
0812 16:51:52.714244 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:52.714250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:52.714345 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:52.714461 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:52.715873 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.716748 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.718209 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.718699 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.718825 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.723244 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:52.723250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:52.723254 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:52.723307 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:52.726173 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:52.726760 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:52.726764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:52.726767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:52.726889 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:52.727205 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:52.727208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:52.727210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:52.727607 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:52.727791 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:52.728039 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 16:51:52.728209 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":316,"threads_started":1}
I20260812 16:51:52.731968 12595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:51:52.732594 12595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:52.739252 12897 tablet_service.cc:1467] Processing CreateTablet for tablet 2be4a90d42544edfb440c1d454362474 (DEFAULT_TABLE table=test-table [id=29cb05e4683c4d0899ba1f504cb42016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.739478 13512 tablet_service.cc:1467] Processing CreateTablet for tablet 2be4a90d42544edfb440c1d454362474 (DEFAULT_TABLE table=test-table [id=29cb05e4683c4d0899ba1f504cb42016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.739526 13183 tablet_service.cc:1467] Processing CreateTablet for tablet 2be4a90d42544edfb440c1d454362474 (DEFAULT_TABLE table=test-table [id=29cb05e4683c4d0899ba1f504cb42016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.739781 12897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be4a90d42544edfb440c1d454362474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.739955 13512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be4a90d42544edfb440c1d454362474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.739964 13183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be4a90d42544edfb440c1d454362474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.746899 13923 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Bootstrap starting.
I20260812 16:51:52.747802 13927 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Bootstrap starting.
I20260812 16:51:52.748091 13926 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Bootstrap starting.
I20260812 16:51:52.748979 13923 tablet_bootstrap.cc:654] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.749830 13927 tablet_bootstrap.cc:654] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.749996 13926 tablet_bootstrap.cc:654] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.752532 13927 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: No bootstrap required, opened a new log
I20260812 16:51:52.752652 13923 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: No bootstrap required, opened a new log
I20260812 16:51:52.752700 13927 ts_tablet_manager.cc:1397] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:52.752799 13923 ts_tablet_manager.cc:1397] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 16:51:52.752825 13926 tablet_bootstrap.cc:492] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: No bootstrap required, opened a new log
I20260812 16:51:52.752982 13926 ts_tablet_manager.cc:1397] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:52.753683 13923 raft_consensus.cc:348] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.753650 13927 raft_consensus.cc:348] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.753913 13923 raft_consensus.cc:374] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.753919 13927 raft_consensus.cc:374] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.754007 13927 raft_consensus.cc:729] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.753935 13926 raft_consensus.cc:348] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.754081 13923 raft_consensus.cc:729] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.754177 13926 raft_consensus.cc:374] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.754281 13926 raft_consensus.cc:729] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.754455 13923 consensus_queue.cc:260] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.754628 13927 consensus_queue.cc:260] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.755328 13923 ts_tablet_manager.cc:1428] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:51:52.754606 13926 consensus_queue.cc:260] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.755723 13927 ts_tablet_manager.cc:1428] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:51:52.756798 13926 ts_tablet_manager.cc:1428] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260812 16:51:52.784653 13946 raft_consensus.cc:482] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:52.784930 13946 raft_consensus.cc:504] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.785805 13946 leader_election.cc:288] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:52.786482 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be4a90d42544edfb440c1d454362474" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" is_pre_election: true
I20260812 16:51:52.786501 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be4a90d42544edfb440c1d454362474" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2" is_pre_election: true
I20260812 16:51:52.786784 13238 raft_consensus.cc:2454] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 0.
I20260812 16:51:52.786921 12944 raft_consensus.cc:2454] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 0.
I20260812 16:51:52.787312 13394 leader_election.cc:302] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95; no voters:
I20260812 16:51:52.787590 13946 raft_consensus.cc:2781] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:52.787703 13946 raft_consensus.cc:482] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:52.787774 13946 raft_consensus.cc:3037] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:52.790238 13946 raft_consensus.cc:504] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.790823 13946 leader_election.cc:288] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 election: Requested vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:52.791244 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be4a90d42544edfb440c1d454362474" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
I20260812 16:51:52.791402 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be4a90d42544edfb440c1d454362474" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2"
I20260812 16:51:52.791500 12944 raft_consensus.cc:3037] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:52.791630 13238 raft_consensus.cc:3037] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:52.794726 12944 raft_consensus.cc:2454] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 1.
I20260812 16:51:52.794756 13238 raft_consensus.cc:2454] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 1.
I20260812 16:51:52.795274 13394 leader_election.cc:302] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95; no voters:
I20260812 16:51:52.795689 13946 raft_consensus.cc:2781] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:52.796301 13946 raft_consensus.cc:686] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 LEADER]: Becoming Leader. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Running, Role: LEADER
I20260812 16:51:52.796741 13946 consensus_queue.cc:237] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.800338 12592 catalog_manager.cc:5495] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "36356a8bdf9141f79c1ddce89bbfac95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } health_report { overall_health: HEALTHY } } }
I20260812 16:51:52.825843 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.818582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.8.195.131:45751)
0812 16:51:52.819408 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:52.819414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:52.819509 (+ 95us) server_negotiation.cc:407] Connection header received
0812 16:51:52.819617 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:52.819623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:52.819730 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:52.819938 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:52.820420 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.821754 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.823270 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.823694 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.823829 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.824405 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:52.825104 (+ 699us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:52.825287 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:52.825342 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 16:51:52.825426 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":431,"threads_started":1}
I20260812 16:51:52.841917 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.836084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:43505 (local address 127.8.195.130:41623)
0812 16:51:52.836672 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:52.836677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:52.837138 (+ 461us) server_negotiation.cc:407] Connection header received
0812 16:51:52.837374 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:52.837378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:52.837441 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:52.837523 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:52.837973 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.838801 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.840291 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.840694 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.840798 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.841280 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:52.841393 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:52.841468 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:52.841616 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 16:51:52.841674 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":268,"threads_started":1}
I20260812 16:51:52.841964 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.836278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41623 (local address 127.8.195.131:43505)
0812 16:51:52.837037 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:52.837057 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:52.837258 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:52.837549 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:52.837553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:52.837562 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:52.837841 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:52.837849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.838929 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.838933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:52.840128 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:52.840141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.840854 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.840859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:52.840980 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.841029 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:52.841486 (+ 457us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:52.841503 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 16:51:52.841570 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 16:51:52.841667 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":418,"threads_started":1}
I20260812 16:51:52.850100 13238 raft_consensus.cc:1261] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Refusing update from remote peer 36356a8bdf9141f79c1ddce89bbfac95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:52.850221 12944 raft_consensus.cc:1261] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Refusing update from remote peer 36356a8bdf9141f79c1ddce89bbfac95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:52.850792 13948 consensus_queue.cc:1035] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:52.851171 13946 consensus_queue.cc:1035] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:52.879768 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.873671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:38699 (local address 127.8.195.131:45751)
0812 16:51:52.873876 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:52.873885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:52.874452 (+ 567us) server_negotiation.cc:407] Connection header received
0812 16:51:52.874689 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:52.874695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:52.874765 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:52.874836 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:51:52.875152 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.876059 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.877961 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.878556 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.878659 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.879108 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:51:52.879197 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:51:52.879267 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:52.879423 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:51:52.879506 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:51:52.879899 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:52.873654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45751 (local address 127.8.195.130:38699)
0812 16:51:52.874292 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:52.874310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:52.874504 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:52.874843 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:52.874846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:52.874856 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:51:52.875046 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:52.875053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.876219 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.876239 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:52.877834 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:52.877847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:52.878791 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:52.878795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:52.878911 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:52.878945 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:51:52.879290 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:51:52.879299 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:51:52.879364 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 16:51:52.879458 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":328,"threads_started":1}
I20260812 16:51:52.913753 12592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:52.914036 12592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:52.916950 12592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Sending DeleteTablet for 3 replicas of tablet 2be4a90d42544edfb440c1d454362474
I20260812 16:51:52.917848 13183 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be4a90d42544edfb440c1d454362474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35900
I20260812 16:51:52.917874 13512 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be4a90d42544edfb440c1d454362474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:33636
I20260812 16:51:52.917956 12897 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be4a90d42544edfb440c1d454362474 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:52 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53154
I20260812 16:51:52.918318 13895 tablet_replica.cc:330] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: stopping tablet replica
I20260812 16:51:52.918529 13895 raft_consensus.cc:2229] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:52.918717 13895 raft_consensus.cc:2258] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:52.918994 13894 tablet_replica.cc:330] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: stopping tablet replica
I20260812 16:51:52.919282 13894 raft_consensus.cc:2229] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:52.919360 13892 tablet_replica.cc:330] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: stopping tablet replica
I20260812 16:51:52.919557 13892 raft_consensus.cc:2229] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:52.919463 12592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 16:51:52.919777 13892 raft_consensus.cc:2258] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:52.919898 13894 raft_consensus.cc:2258] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 16:51:52.920143 12592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:51:52.921320 13894 ts_tablet_manager.cc:1905] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.921674 13895 ts_tablet_manager.cc:1905] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.927680 13894 ts_tablet_manager.cc:1918] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:52.927976 13894 log.cc:1196] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/2be4a90d42544edfb440c1d454362474
I20260812 16:51:52.928683 13894 ts_tablet_manager.cc:1939] T 2be4a90d42544edfb440c1d454362474 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting consensus metadata
I20260812 16:51:52.929163 13892 ts_tablet_manager.cc:1905] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:52.929996 12570 catalog_manager.cc:4841] TS 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751): tablet 2be4a90d42544edfb440c1d454362474 (table test-table [id=29cb05e4683c4d0899ba1f504cb42016]) successfully deleted
I20260812 16:51:52.931546 12897 tablet_service.cc:1467] Processing CreateTablet for tablet 0b66d0204770494c9bc49c5be9559c33 (DEFAULT_TABLE table=test-table [id=c7736ec7462c4d148241ad67332752aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.931718 13183 tablet_service.cc:1467] Processing CreateTablet for tablet 0b66d0204770494c9bc49c5be9559c33 (DEFAULT_TABLE table=test-table [id=c7736ec7462c4d148241ad67332752aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.932161 13183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b66d0204770494c9bc49c5be9559c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.932250 12897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b66d0204770494c9bc49c5be9559c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.932994 13512 tablet_service.cc:1467] Processing CreateTablet for tablet 0b66d0204770494c9bc49c5be9559c33 (DEFAULT_TABLE table=test-table [id=c7736ec7462c4d148241ad67332752aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:51:52.933553 13512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b66d0204770494c9bc49c5be9559c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:52.936856 13895 ts_tablet_manager.cc:1918] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:52.937402 13895 log.cc:1196] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/2be4a90d42544edfb440c1d454362474
I20260812 16:51:52.938537 13895 ts_tablet_manager.cc:1939] T 2be4a90d42544edfb440c1d454362474 P 17248d0a4a2943a79a312c8f350a61f2: Deleting consensus metadata
I20260812 16:51:52.939216 13926 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Bootstrap starting.
I20260812 16:51:52.940254 12567 catalog_manager.cc:4841] TS 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623): tablet 2be4a90d42544edfb440c1d454362474 (table test-table [id=29cb05e4683c4d0899ba1f504cb42016]) successfully deleted
I20260812 16:51:52.941278 13926 tablet_bootstrap.cc:654] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.942497 13927 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Bootstrap starting.
I20260812 16:51:52.944989 13927 tablet_bootstrap.cc:654] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.946805 13892 ts_tablet_manager.cc:1918] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:51:52.947014 13892 log.cc:1196] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/2be4a90d42544edfb440c1d454362474
I20260812 16:51:52.947604 13892 ts_tablet_manager.cc:1939] T 2be4a90d42544edfb440c1d454362474 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting consensus metadata
I20260812 16:51:52.948297 13923 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Bootstrap starting.
I20260812 16:51:52.951050 13923 tablet_bootstrap.cc:654] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:52.956511 13926 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: No bootstrap required, opened a new log
I20260812 16:51:52.956879 13926 ts_tablet_manager.cc:1397] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.012s
I20260812 16:51:52.958194 13926 raft_consensus.cc:348] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.958346 12568 catalog_manager.cc:4841] TS cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415): tablet 2be4a90d42544edfb440c1d454362474 (table test-table [id=29cb05e4683c4d0899ba1f504cb42016]) successfully deleted
I20260812 16:51:52.958501 13926 raft_consensus.cc:374] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.958590 13926 raft_consensus.cc:729] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17248d0a4a2943a79a312c8f350a61f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.958945 13926 consensus_queue.cc:260] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.960688 13926 ts_tablet_manager.cc:1428] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 16:51:52.962463 13927 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: No bootstrap required, opened a new log
I20260812 16:51:52.962669 13927 ts_tablet_manager.cc:1397] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.011s
I20260812 16:51:52.963307 13923 tablet_bootstrap.cc:492] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: No bootstrap required, opened a new log
I20260812 16:51:52.963510 13923 ts_tablet_manager.cc:1397] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20260812 16:51:52.963551 13927 raft_consensus.cc:348] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.963843 13927 raft_consensus.cc:374] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.963936 13927 raft_consensus.cc:729] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.964253 13927 consensus_queue.cc:260] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.964354 13923 raft_consensus.cc:348] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.964586 13923 raft_consensus.cc:374] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:52.964687 13923 raft_consensus.cc:729] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3042f47e3e4a98b9a5f50c42a5c2f2, State: Initialized, Role: FOLLOWER
I20260812 16:51:52.965036 13927 ts_tablet_manager.cc:1428] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 16:51:52.964929 13923 consensus_queue.cc:260] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:52.965842 13923 ts_tablet_manager.cc:1428] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:51:53.075848 13946 raft_consensus.cc:482] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:51:53.076110 13946 raft_consensus.cc:504] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:53.076925 13946 leader_election.cc:288] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:53.077905 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b66d0204770494c9bc49c5be9559c33" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" is_pre_election: true
I20260812 16:51:53.077880 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b66d0204770494c9bc49c5be9559c33" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2" is_pre_election: true
I20260812 16:51:53.078171 13238 raft_consensus.cc:2454] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 0.
I20260812 16:51:53.078488 12944 raft_consensus.cc:2454] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 0.
I20260812 16:51:53.078732 13394 leader_election.cc:302] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17248d0a4a2943a79a312c8f350a61f2, 36356a8bdf9141f79c1ddce89bbfac95; no voters:
I20260812 16:51:53.079070 13946 raft_consensus.cc:2781] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:51:53.079258 13946 raft_consensus.cc:482] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:51:53.079447 13946 raft_consensus.cc:3037] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:53.082034 13946 raft_consensus.cc:504] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:53.083182 12944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b66d0204770494c9bc49c5be9559c33" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2"
I20260812 16:51:53.083184 13238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b66d0204770494c9bc49c5be9559c33" candidate_uuid: "36356a8bdf9141f79c1ddce89bbfac95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17248d0a4a2943a79a312c8f350a61f2"
I20260812 16:51:53.083429 13238 raft_consensus.cc:3037] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:53.083573 12944 raft_consensus.cc:3037] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:53.087204 12944 raft_consensus.cc:2454] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 1.
I20260812 16:51:53.087584 13238 raft_consensus.cc:2454] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36356a8bdf9141f79c1ddce89bbfac95 in term 1.
I20260812 16:51:53.087792 13395 leader_election.cc:302] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36356a8bdf9141f79c1ddce89bbfac95, cb3042f47e3e4a98b9a5f50c42a5c2f2; no voters:
I20260812 16:51:53.088084 13946 leader_election.cc:288] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [CANDIDATE]: Term 1 election: Requested vote from peers cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415), 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623)
I20260812 16:51:53.088150 13948 raft_consensus.cc:2781] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:53.088325 13948 raft_consensus.cc:686] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 LEADER]: Becoming Leader. State: Replica: 36356a8bdf9141f79c1ddce89bbfac95, State: Running, Role: LEADER
I20260812 16:51:53.088637 13948 consensus_queue.cc:237] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } }
I20260812 16:51:53.091943 12592 catalog_manager.cc:5495] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "36356a8bdf9141f79c1ddce89bbfac95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36356a8bdf9141f79c1ddce89bbfac95" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45751 } health_report { overall_health: HEALTHY } } }
I20260812 16:51:53.129967 13238 raft_consensus.cc:1261] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Refusing update from remote peer 36356a8bdf9141f79c1ddce89bbfac95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:53.130003 12944 raft_consensus.cc:1261] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Refusing update from remote peer 36356a8bdf9141f79c1ddce89bbfac95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:51:53.130743 13973 consensus_queue.cc:1035] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3042f47e3e4a98b9a5f50c42a5c2f2" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:51:53.131242 13948 consensus_queue.cc:1035] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17248d0a4a2943a79a312c8f350a61f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:51:53.167235 13857 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 16:51:54.770435 12592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:54.770673 12592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:51:54.775152 12592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82a0e617e481460985eba58df12c97a4: Sending DeleteTablet for 3 replicas of tablet 0b66d0204770494c9bc49c5be9559c33
I20260812 16:51:54.776113 12897 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b66d0204770494c9bc49c5be9559c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:54 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53154
I20260812 16:51:54.776098 13183 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b66d0204770494c9bc49c5be9559c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:54 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35900
I20260812 16:51:54.776218 13512 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b66d0204770494c9bc49c5be9559c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:51:54 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:33636
I20260812 16:51:54.777101 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12757
I20260812 16:51:54.810007 14041 tablet_replica.cc:330] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: stopping tablet replica
I20260812 16:51:54.810371 14041 raft_consensus.cc:2229] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:54.810447 14042 tablet_replica.cc:330] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: stopping tablet replica
I20260812 16:51:54.810758 14042 raft_consensus.cc:2229] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:51:54.810985 14041 raft_consensus.cc:2258] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:54.811283 14042 raft_consensus.cc:2258] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:54.811972 14040 tablet_replica.cc:330] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: stopping tablet replica
I20260812 16:51:54.812274 14040 raft_consensus.cc:2229] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:51:54.812523 14041 ts_tablet_manager.cc:1905] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:54.812577 14040 raft_consensus.cc:2258] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:51:54.812951 14042 ts_tablet_manager.cc:1905] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:54.814033 14040 ts_tablet_manager.cc:1905] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:51:54.818926 14042 ts_tablet_manager.cc:1918] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:54.819089 14042 log.cc:1196] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/0b66d0204770494c9bc49c5be9559c33
I20260812 16:51:54.819561 14042 ts_tablet_manager.cc:1939] T 0b66d0204770494c9bc49c5be9559c33 P 36356a8bdf9141f79c1ddce89bbfac95: Deleting consensus metadata
Aug 12 16:51:54 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:51:54.820758 12570 catalog_manager.cc:4841] TS 36356a8bdf9141f79c1ddce89bbfac95 (127.8.195.131:45751): tablet 0b66d0204770494c9bc49c5be9559c33 (table test-table [id=c7736ec7462c4d148241ad67332752aa]) successfully deleted
I20260812 16:51:54.820828 14041 ts_tablet_manager.cc:1918] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:54.821010 14041 log.cc:1196] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/0b66d0204770494c9bc49c5be9559c33
I20260812 16:51:54.821676 14041 ts_tablet_manager.cc:1939] T 0b66d0204770494c9bc49c5be9559c33 P cb3042f47e3e4a98b9a5f50c42a5c2f2: Deleting consensus metadata
I20260812 16:51:54.822387 14040 ts_tablet_manager.cc:1918] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 16:51:54.822655 14040 log.cc:1196] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/0b66d0204770494c9bc49c5be9559c33
I20260812 16:51:54.822751 12568 catalog_manager.cc:4841] TS cb3042f47e3e4a98b9a5f50c42a5c2f2 (127.8.195.129:35415): tablet 0b66d0204770494c9bc49c5be9559c33 (table test-table [id=c7736ec7462c4d148241ad67332752aa]) successfully deleted
I20260812 16:51:54.823470 14040 ts_tablet_manager.cc:1939] T 0b66d0204770494c9bc49c5be9559c33 P 17248d0a4a2943a79a312c8f350a61f2: Deleting consensus metadata
I20260812 16:51:54.824806 12567 catalog_manager.cc:4841] TS 17248d0a4a2943a79a312c8f350a61f2 (127.8.195.130:41623): tablet 0b66d0204770494c9bc49c5be9559c33 (table test-table [id=c7736ec7462c4d148241ad67332752aa]) successfully deleted
I20260812 16:51:54.828912 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:54.780491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.8.195.129:35415)
0812 16:51:54.802417 (+ 21926us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:54.802428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:54.803455 (+ 1027us) server_negotiation.cc:407] Connection header received
0812 16:51:54.803578 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:54.803587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:54.803698 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:54.803839 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:54.806335 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.807934 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.811511 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.812150 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.812252 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:54.821810 (+ 9558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:54.821832 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:54.821854 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:54.821899 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:54.826121 (+ 4222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.826788 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.826795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.826797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.827007 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.827408 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.827411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.827413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.827870 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:54.828175 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:54.828360 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 16:51:54.828532 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21684,"thread_start_us":21957,"threads_started":1}
W20260812 16:51:54.829710 13039 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40762
W20260812 16:51:54.830364 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12757: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:54.830590 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 12757
I20260812 16:51:54.866840 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13054
Aug 12 16:51:54 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:51:54.883404 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:54.867640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.8.195.130:41623)
0812 16:51:54.868200 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:54.868205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:54.868220 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:51:54.868273 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:54.868277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:54.868342 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:54.868462 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:54.869994 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.870797 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.872095 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.872717 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.872877 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:54.877602 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:54.877616 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:54.877631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:54.877668 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:54.880560 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.881267 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.881275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.881277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.881421 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.881825 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.881828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.881830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.882188 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:54.882476 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:54.882820 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 16:51:54.883060 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":267,"threads_started":1}
W20260812 16:51:54.884114 13347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54196
W20260812 16:51:54.884733 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13054: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:54.884897 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13054
I20260812 16:51:54.915660 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13365
Aug 12 16:51:54 dist-test-slave-z8x9 krb5kdc[12502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:51:54.931769 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:54.916421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.8.195.131:45751)
0812 16:51:54.916885 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:54.916889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:54.916905 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:51:54.916962 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:54.916965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:54.917024 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:54.917113 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:54.918917 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.919667 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.920982 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:54.921527 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:54.921631 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:54.925918 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:54.925938 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:54.925955 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:54.925994 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:54.929006 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.929826 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.929835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.929837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.929974 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:54.930348 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:54.930351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:54.930353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:54.930710 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:54.930930 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:54.931201 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 16:51:54.931367 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
W20260812 16:51:54.932458 13666 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55058
W20260812 16:51:54.933068 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13365: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:51:54.933317 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13365
I20260812 16:51:54.975836 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12524
I20260812 16:51:54.977214 12703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:55.097616 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 12524
2026-08-12T16:51:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8760 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:51:55 dist-test-slave-z8x9 krb5kdc[14055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:51:55 dist-test-slave-z8x9 krb5kdc[14055](info): set up 2 sockets
Aug 12 16:51:55 dist-test-slave-z8x9 krb5kdc[14055](info): commencing operation
krb5kdc: starting...
W20260812 16:51:56.480875 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:51:56 dist-test-slave-z8x9 krb5kdc[14055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:51:56.720918 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39083
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:39083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:57.055511 14077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:57.055835 14077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:57.056007 14077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:57.063126 14077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:51:57.063282 14077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:57.063387 14077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:57.063491 14077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:51:57.063578 14077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:51:57.073060 14077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:39083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:57.074370 14077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:57.075654 14077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:57.088039 14086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:57.088153 14077 server_base.cc:1034] running on GCE node
W20260812 16:51:57.088106 14092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:57.088550 14085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:57.089548 14077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:57.090893 14077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:57.092103 14077 hybrid_clock.cc:648] HybridClock initialized: now 1786553517092049 us; error 37 us; skew 500 ppm
I20260812 16:51:57.092409 14077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:57 dist-test-slave-z8x9 krb5kdc[14055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553517, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:57.095526 14077 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:51:57.097059 14077 webserver.cc:460] Webserver started at http://127.8.195.190:41601/ using document root <none> and password file <none>
I20260812 16:51:57.097774 14077 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:57.097880 14077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:57.098160 14077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:57.101046 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "e264ebdbffd744418b9063456e75e456"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "7d82fee4dd3b1f85652f75b902d13e01"
server_key_iv: "9da29f44e81932a47988721e73ff9646"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.101866 14077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "e264ebdbffd744418b9063456e75e456"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "7d82fee4dd3b1f85652f75b902d13e01"
server_key_iv: "9da29f44e81932a47988721e73ff9646"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.107030 14077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:51:57.110437 14109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:57.111797 14077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 16:51:57.111982 14077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "e264ebdbffd744418b9063456e75e456"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "7d82fee4dd3b1f85652f75b902d13e01"
server_key_iv: "9da29f44e81932a47988721e73ff9646"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.112200 14077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:57.132359 14077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:57.136832 14077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:57.137162 14077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:57.156891 14077 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:39083
I20260812 16:51:57.156924 14219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:39083 every 8 connection(s)
I20260812 16:51:57.158466 14077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:51:57.162544 14220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:51:57.163477 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14077
I20260812 16:51:57.163746 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:51:57.164609 8974 external_mini_cluster.cc:1428] Setting key 57a8d4cef71135af4f055f9328fb142b
I20260812 16:51:57.177970 14220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: Bootstrap starting.
Aug 12 16:51:57 dist-test-slave-z8x9 krb5kdc[14055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:57.182116 14220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: Neither blocks nor log segments found. Creating new log.
I20260812 16:51:57.183645 14220 log.cc:824] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: Log is configured to *not* fsync() on all Append() calls
I20260812 16:51:57.187242 14220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: No bootstrap required, opened a new log
I20260812 16:51:57.190341 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:57.166868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.8.195.190:39083)
0812 16:51:57.167713 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:57.167725 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:57.167776 (+ 51us) server_negotiation.cc:407] Connection header received
0812 16:51:57.169541 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:57.169569 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:57.170015 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:57.170360 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:57.172333 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.173730 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.175622 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.176321 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.176467 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:57.181472 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:57.181483 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:57.181502 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:57.181552 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:57.185599 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:57.186321 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:57.186329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:57.186341 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:57.186553 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:57.186920 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:57.186925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:57.186928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:57.187731 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:57.188048 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:57.188371 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 16:51:57.188843 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":393,"threads_started":1}
I20260812 16:51:57.196542 14220 raft_consensus.cc:348] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } }
I20260812 16:51:57.196923 14220 raft_consensus.cc:374] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:51:57.197050 14220 raft_consensus.cc:729] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e264ebdbffd744418b9063456e75e456, State: Initialized, Role: FOLLOWER
I20260812 16:51:57.197767 14220 consensus_queue.cc:260] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } }
I20260812 16:51:57.198025 14220 raft_consensus.cc:388] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:51:57.198217 14220 raft_consensus.cc:482] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:51:57.198405 14220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:51:57.200845 14220 raft_consensus.cc:504] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } }
I20260812 16:51:57.201423 14220 leader_election.cc:302] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e264ebdbffd744418b9063456e75e456; no voters:
I20260812 16:51:57.202026 14220 leader_election.cc:288] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:51:57.202240 14232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:51:57.202978 14232 raft_consensus.cc:686] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [term 1 LEADER]: Becoming Leader. State: Replica: e264ebdbffd744418b9063456e75e456, State: Running, Role: LEADER
I20260812 16:51:57.203568 14232 consensus_queue.cc:237] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } }
I20260812 16:51:57.204165 14220 sys_catalog.cc:564] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:51:57.206727 14237 sys_catalog.cc:455] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e264ebdbffd744418b9063456e75e456. Latest consensus state: current_term: 1 leader_uuid: "e264ebdbffd744418b9063456e75e456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } } }
I20260812 16:51:57.207122 14237 sys_catalog.cc:458] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [sys.catalog]: This master's current role is: LEADER
I20260812 16:51:57.206751 14234 sys_catalog.cc:455] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e264ebdbffd744418b9063456e75e456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e264ebdbffd744418b9063456e75e456" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39083 } } }
I20260812 16:51:57.209100 14234 sys_catalog.cc:458] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456 [sys.catalog]: This master's current role is: LEADER
W20260812 16:51:57.221585 14267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:51:57.221756 14267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:51:57.221918 14248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:51:57.223837 14248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:51:57.232306 14248 catalog_manager.cc:1347] Generated new cluster ID: d1f6645bafb24aeebbda6700151c8809
I20260812 16:51:57.232425 14248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:51:57.248885 14248 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:51:57.250901 14248 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:51:57.263727 14248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e264ebdbffd744418b9063456e75e456: Generated new TSK 0
I20260812 16:51:57.264716 14248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:51:57.334656 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:57.706842 14273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:57.707181 14273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:57.707283 14273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:57.715258 14273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:57.715394 14273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:57.715565 14273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:51:57.725840 14273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:57.727170 14273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:57.728427 14273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:57.741926 14279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:57.741957 14281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:57.741920 14286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:57.743202 14273 server_base.cc:1034] running on GCE node
I20260812 16:51:57.743902 14273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:57.744906 14273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:57.746111 14273 hybrid_clock.cc:648] HybridClock initialized: now 1786553517746029 us; error 73 us; skew 500 ppm
I20260812 16:51:57.746455 14273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:57 dist-test-slave-z8x9 krb5kdc[14055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553517, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:57.749825 14273 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:51:57.751384 14273 webserver.cc:460] Webserver started at http://127.8.195.129:42473/ using document root <none> and password file <none>
I20260812 16:51:57.752085 14273 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:57.752198 14273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:57.752447 14273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:57.755353 14273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "be56fa17f21e4011a47a96eb6c702df7"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "2a9e40348f2d43ef49e4cea090587351"
server_key_iv: "b8fdaffaf7f6c996fae6af35f6cacfb7"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.756099 14273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "be56fa17f21e4011a47a96eb6c702df7"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "2a9e40348f2d43ef49e4cea090587351"
server_key_iv: "b8fdaffaf7f6c996fae6af35f6cacfb7"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.761631 14273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:51:57.765370 14299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:57.766901 14273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:51:57.767050 14273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "be56fa17f21e4011a47a96eb6c702df7"
format_stamp: "Formatted at 2026-08-12 16:51:57 on dist-test-slave-z8x9"
server_key: "2a9e40348f2d43ef49e4cea090587351"
server_key_iv: "b8fdaffaf7f6c996fae6af35f6cacfb7"
server_key_version: "encryptionkey@0"
I20260812 16:51:57.767228 14273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:57.794232 14273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:57.798712 14273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:57.798966 14273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:57.799934 14273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:57.801323 14273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:57.801401 14273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:57.801488 14273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:57.801533 14273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:57.842324 14273 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:44833
I20260812 16:51:57.842365 14522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:44833 every 8 connection(s)
I20260812 16:51:57.843640 14273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:51:57.851608 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14273
I20260812 16:51:57.851847 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:51:57.852521 8974 external_mini_cluster.cc:1428] Setting key 00b46a1ea50769c563cee48aba72597b
Aug 12 16:51:57 dist-test-slave-z8x9 krb5kdc[14055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553517, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:57.865159 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:57.846386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:48595 (local address 127.8.195.190:39083)
0812 16:51:57.846877 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:57.846881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:57.847655 (+ 774us) server_negotiation.cc:407] Connection header received
0812 16:51:57.849617 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:57.849623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:57.849721 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:57.849843 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:57.851412 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.852259 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.854066 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.854638 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.854779 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:57.858706 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:57.858712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:57.858715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:57.858750 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:57.862008 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:57.862792 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:57.862797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:57.862800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:57.862920 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:57.863379 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:57.863389 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:57.863391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:57.863664 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:57.863822 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:57.864545 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 16:51:57.864778 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":267,"threads_started":1}
I20260812 16:51:57.866196 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:57.846762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39083 (local address 127.8.195.129:48595)
0812 16:51:57.847478 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:57.847517 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:57.849329 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:57.850095 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:57.850103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:57.850635 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:57.851230 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:57.851249 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.852427 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.852431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:57.853881 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:57.853899 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:57.854885 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:57.854893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:57.855121 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:57.855857 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:57.855880 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:57.858434 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:57.862167 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:57.862176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:57.862193 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:57.862634 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:57.863039 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:57.863043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:57.863048 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:57.863248 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:57.863797 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:57.863805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:57.864156 (+ 351us) client_negotiation.cc:769] Sending connection context
0812 16:51:57.864427 (+ 271us) client_negotiation.cc:240] Negotiation successful
0812 16:51:57.864983 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":288,"threads_started":1}
I20260812 16:51:57.867913 14523 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39083
I20260812 16:51:57.868232 14523 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:57.868937 14523 heartbeater.cc:511] Master 127.8.195.190:39083 requested a full tablet report, sending...
I20260812 16:51:57.871482 14139 ts_manager.cc:194] Registered new tserver with Master: be56fa17f21e4011a47a96eb6c702df7 (127.8.195.129:44833)
I20260812 16:51:57.873772 14139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:48595
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:51:57.904812 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:58.239035 14532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:58.239372 14532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:58.239480 14532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:58.246798 14532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:58.246932 14532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:58.247049 14532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:51:58.256984 14532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:58.258406 14532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:58.259780 14532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:58.274132 14544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:58.274217 14540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:58.274336 14547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:58.274825 14532 server_base.cc:1034] running on GCE node
I20260812 16:51:58.275460 14532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:58.276425 14532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:58.277652 14532 hybrid_clock.cc:648] HybridClock initialized: now 1786553518277594 us; error 52 us; skew 500 ppm
I20260812 16:51:58.277973 14532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:58 dist-test-slave-z8x9 krb5kdc[14055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553518, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:58.281260 14532 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:51:58.282799 14532 webserver.cc:460] Webserver started at http://127.8.195.130:36683/ using document root <none> and password file <none>
I20260812 16:51:58.283463 14532 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:58.283567 14532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:58.283844 14532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:58.286986 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "704a9c67651948f39e5c1a7276ca8cb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "e917ec458b439c0b6a45bdaa9e93f320"
server_key_iv: "343cffe0cfb2e301b762fa42026b3d3d"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.287741 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "704a9c67651948f39e5c1a7276ca8cb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "e917ec458b439c0b6a45bdaa9e93f320"
server_key_iv: "343cffe0cfb2e301b762fa42026b3d3d"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.292909 14532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 16:51:58.296420 14562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.297796 14532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:51:58.297964 14532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "704a9c67651948f39e5c1a7276ca8cb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "e917ec458b439c0b6a45bdaa9e93f320"
server_key_iv: "343cffe0cfb2e301b762fa42026b3d3d"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.298146 14532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:58.323055 14532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:58.327500 14532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:58.327821 14532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:58.328862 14532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:58.330195 14532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:58.330272 14532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.330435 14532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:58.330513 14532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.365482 14532 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:34179
I20260812 16:51:58.365609 14790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:34179 every 8 connection(s)
I20260812 16:51:58.367244 14532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:51:58.377559 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14532
I20260812 16:51:58.377780 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:51:58.378204 8974 external_mini_cluster.cc:1428] Setting key c33dc66fa169b621406f9780b4b9d90a
Aug 12 16:51:58 dist-test-slave-z8x9 krb5kdc[14055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553518, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:58.388300 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:58.369888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:46941 (local address 127.8.195.190:39083)
0812 16:51:58.370430 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:58.370434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:58.371293 (+ 859us) server_negotiation.cc:407] Connection header received
0812 16:51:58.373348 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:58.373354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:58.373445 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:58.373594 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:58.375166 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.375924 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.377603 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.378180 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.378317 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:58.382388 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:58.382394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:58.382397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:58.382434 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:58.385111 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:58.385918 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:58.385924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:58.385928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:58.386070 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:58.386527 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:58.386532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:58.386534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:58.386816 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:58.386976 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:58.387676 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 16:51:58.387903 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":248,"threads_started":1}
I20260812 16:51:58.389387 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:58.370440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39083 (local address 127.8.195.130:46941)
0812 16:51:58.371117 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:58.371167 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:58.372994 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:58.373842 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:58.373850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:58.374366 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:58.374979 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:58.374996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.376081 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.376091 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:58.377447 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:58.377465 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.378421 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.378428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:58.378672 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:58.379534 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:58.379564 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:58.382163 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:58.385317 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:58.385326 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:58.385343 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:58.385786 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:58.386194 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:58.386200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:58.386203 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:58.386390 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:58.386949 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:58.386957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:58.387310 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 16:51:58.387579 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 16:51:58.388135 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":255,"threads_started":1}
I20260812 16:51:58.390924 14793 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39083
I20260812 16:51:58.391278 14793 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:58.392172 14793 heartbeater.cc:511] Master 127.8.195.190:39083 requested a full tablet report, sending...
I20260812 16:51:58.394007 14139 ts_manager.cc:194] Registered new tserver with Master: 704a9c67651948f39e5c1a7276ca8cb2 (127.8.195.130:34179)
I20260812 16:51:58.395296 14139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:46941
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:51:58.432337 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:51:58.778460 14810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:51:58.778800 14810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:51:58.778985 14810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:51:58.786636 14810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:51:58.786773 14810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:51:58.786974 14810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:51:58.797356 14810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:51:58.798749 14810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:51:58.800120 14810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:51:58.814801 14821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:58.814824 14825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:51:58.814927 14827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:51:58.815721 14810 server_base.cc:1034] running on GCE node
I20260812 16:51:58.816334 14810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:51:58.817446 14810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:51:58.818684 14810 hybrid_clock.cc:648] HybridClock initialized: now 1786553518818651 us; error 58 us; skew 500 ppm
I20260812 16:51:58.819023 14810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:51:58 dist-test-slave-z8x9 krb5kdc[14055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553518, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:51:58.822197 14810 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:51:58.823735 14810 webserver.cc:460] Webserver started at http://127.8.195.131:45671/ using document root <none> and password file <none>
I20260812 16:51:58.824442 14810 fs_manager.cc:359] Metadata directory not provided
I20260812 16:51:58.824595 14810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:51:58.824919 14810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:51:58.827993 14810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "a412548a6ab14dfa9f4d6e6d0040fcb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "9546a4a65ba380b330afcc41c67cce38"
server_key_iv: "1732458100b4c8f9435621b3afa54cc5"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.828785 14810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "a412548a6ab14dfa9f4d6e6d0040fcb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "9546a4a65ba380b330afcc41c67cce38"
server_key_iv: "1732458100b4c8f9435621b3afa54cc5"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.833937 14810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 16:51:58.837484 14843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.838779 14810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:51:58.838980 14810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "a412548a6ab14dfa9f4d6e6d0040fcb2"
format_stamp: "Formatted at 2026-08-12 16:51:58 on dist-test-slave-z8x9"
server_key: "9546a4a65ba380b330afcc41c67cce38"
server_key_iv: "1732458100b4c8f9435621b3afa54cc5"
server_key_version: "encryptionkey@0"
I20260812 16:51:58.839210 14810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:51:58.852336 14810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:51:58.856469 14810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:51:58.856777 14810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:51:58.857857 14810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:51:58.859261 14810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:51:58.859375 14810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.859529 14810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:51:58.859606 14810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:51:58.877045 14523 heartbeater.cc:503] Master 127.8.195.190:39083 was elected leader, sending a full tablet report...
I20260812 16:51:58.897718 14810 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:46041
I20260812 16:51:58.897775 15099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:46041 every 8 connection(s)
I20260812 16:51:58.899061 14810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:51:58.906376 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14810
I20260812 16:51:58.906594 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:51:58.906996 8974 external_mini_cluster.cc:1428] Setting key bf6c8e8c7189aa991a85e66bec56e412
Aug 12 16:51:58 dist-test-slave-z8x9 krb5kdc[14055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553518, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:51:58.920143 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:58.901802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:60997 (local address 127.8.195.190:39083)
0812 16:51:58.902315 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:58.902320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:58.903291 (+ 971us) server_negotiation.cc:407] Connection header received
0812 16:51:58.905243 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:58.905249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:58.905355 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:58.905479 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:58.907138 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.908081 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.909741 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.910206 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.910344 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:58.914004 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:51:58.914010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:51:58.914013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:51:58.914044 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:51:58.917001 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:58.917785 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:58.917793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:58.917797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:58.917934 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:51:58.918371 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:51:58.918376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:51:58.918378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:51:58.918669 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:51:58.918836 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 16:51:58.919536 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 16:51:58.919754 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":259,"threads_started":1}
I20260812 16:51:58.921103 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:51:58.902354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39083 (local address 127.8.195.131:60997)
0812 16:51:58.903118 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 16:51:58.903165 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:51:58.904937 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:51:58.905726 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:51:58.905735 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:51:58.906282 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:51:58.906955 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:58.906973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.908247 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.908252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:58.909577 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:51:58.909594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:51:58.910445 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:51:58.910452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:51:58.910694 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:51:58.911497 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:51:58.911521 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:51:58.913794 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:51:58.917158 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:58.917192 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:58.917208 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:58.917644 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:58.918048 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:51:58.918053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:51:58.918057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:51:58.918248 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:51:58.918800 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:51:58.918808 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:51:58.919159 (+ 351us) client_negotiation.cc:769] Sending connection context
0812 16:51:58.919429 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 16:51:58.919968 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":305,"threads_started":1}
I20260812 16:51:58.922593 15100 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39083
I20260812 16:51:58.922968 15100 heartbeater.cc:464] Registering TS with master...
I20260812 16:51:58.923663 15100 heartbeater.cc:511] Master 127.8.195.190:39083 requested a full tablet report, sending...
I20260812 16:51:58.924993 14139 ts_manager.cc:194] Registered new tserver with Master: a412548a6ab14dfa9f4d6e6d0040fcb2 (127.8.195.131:46041)
I20260812 16:51:58.925845 14139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:60997
I20260812 16:51:58.935400 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:51:58.979110 15109 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:51:58.976274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.8.195.190:39083)
0812 16:51:58.976479 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 16:51:58.976484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:51:58.976903 (+ 419us) server_negotiation.cc:407] Connection header received
0812 16:51:58.977108 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:51:58.977113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:51:58.977394 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:51:58.977495 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:51:58.978892 (+ 1397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51010: BlockingRecv error: recv got EOF from 127.0.0.1:51010 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:51:58.982858 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14273
W20260812 16:51:58.986063 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14273: Not authorized: Client connection negotiation failed: client connection to 127.8.195.129:44833: server requires authentication, but client does not have Kerberos credentials available
I20260812 16:51:58.986269 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14273
I20260812 16:51:59.023658 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14532
W20260812 16:51:59.026513 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14532: Not authorized: Client connection negotiation failed: client connection to 127.8.195.130:34179: server requires authentication, but client does not have Kerberos credentials available
I20260812 16:51:59.026703 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14532
I20260812 16:51:59.051975 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14810
W20260812 16:51:59.054998 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14810: Not authorized: Client connection negotiation failed: client connection to 127.8.195.131:46041: server requires authentication, but client does not have Kerberos credentials available
I20260812 16:51:59.055131 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14810
I20260812 16:51:59.081377 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14077
I20260812 16:51:59.082692 14213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:51:59.188122 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14077
2026-08-12T16:51:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4094 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:00 dist-test-slave-z8x9 krb5kdc[15139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:00 dist-test-slave-z8x9 krb5kdc[15139](info): set up 2 sockets
Aug 12 16:52:00 dist-test-slave-z8x9 krb5kdc[15139](info): commencing operation
krb5kdc: starting...
W20260812 16:52:01.706745 8974 mini_kdc.cc:121] Time spent starting KDC: real 2.461s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:01 dist-test-slave-z8x9 krb5kdc[15139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:01.931108 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34961
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:02.262486 15168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:02.262802 15168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:02.262988 15168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:02.270354 15168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:02.270483 15168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:02.270610 15168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:02.270721 15168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:02.270815 15168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:02.281002 15168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:02.282414 15168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:02.283776 15168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:02.295473 15175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:02.295449 15179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:02.295617 15176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:02.296921 15168 server_base.cc:1034] running on GCE node
I20260812 16:52:02.297863 15168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:02.299263 15168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:02.300477 15168 hybrid_clock.cc:648] HybridClock initialized: now 1786553522300432 us; error 41 us; skew 500 ppm
I20260812 16:52:02.300836 15168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:02 dist-test-slave-z8x9 krb5kdc[15139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:02.304248 15168 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:02.305871 15168 webserver.cc:460] Webserver started at http://127.8.195.190:34639/ using document root <none> and password file <none>
I20260812 16:52:02.306565 15168 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:02.306701 15168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:02.307034 15168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:02.310132 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "f1ff77c2ed5b4810aab0d814e16410a3"
format_stamp: "Formatted at 2026-08-12 16:52:02 on dist-test-slave-z8x9"
server_key: "216a284e87762457600fdedcc64c18fd"
server_key_iv: "7f708ee74590ab688d73cdd3c8b1718c"
server_key_version: "encryptionkey@0"
I20260812 16:52:02.310956 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "f1ff77c2ed5b4810aab0d814e16410a3"
format_stamp: "Formatted at 2026-08-12 16:52:02 on dist-test-slave-z8x9"
server_key: "216a284e87762457600fdedcc64c18fd"
server_key_iv: "7f708ee74590ab688d73cdd3c8b1718c"
server_key_version: "encryptionkey@0"
I20260812 16:52:02.316295 15168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:52:02.319816 15189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:02.321158 15168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:02.321398 15168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "f1ff77c2ed5b4810aab0d814e16410a3"
format_stamp: "Formatted at 2026-08-12 16:52:02 on dist-test-slave-z8x9"
server_key: "216a284e87762457600fdedcc64c18fd"
server_key_iv: "7f708ee74590ab688d73cdd3c8b1718c"
server_key_version: "encryptionkey@0"
I20260812 16:52:02.321602 15168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:02.347400 15168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:02.351640 15168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:02.351949 15168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:02.371236 15168 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34961
I20260812 16:52:02.371284 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34961 every 8 connection(s)
I20260812 16:52:02.372655 15168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:02.373876 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15168
I20260812 16:52:02.374095 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:02.374804 8974 external_mini_cluster.cc:1428] Setting key 0b400264ad5c0e7d4a25f4f6ec6632d7
I20260812 16:52:02.377038 15317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:52:02 dist-test-slave-z8x9 krb5kdc[15139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:02.390630 15317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3: Bootstrap starting.
I20260812 16:52:02.394865 15317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:02.396230 15317 log.cc:824] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:02.399772 15317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3: No bootstrap required, opened a new log
I20260812 16:52:02.400826 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:02.376673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.8.195.190:34961)
0812 16:52:02.377589 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:02.377605 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:02.377655 (+ 50us) server_negotiation.cc:407] Connection header received
0812 16:52:02.379994 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:02.380022 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:02.380531 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:02.380979 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:02.382423 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:02.384571 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:02.385937 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:02.387013 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:02.387221 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:02.391637 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:02.391649 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:02.391671 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:02.391718 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:02.395896 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:02.396720 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:02.396735 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:02.396747 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:02.397016 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:02.397547 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:02.397551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:02.397553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:02.398310 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:02.398620 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:02.399034 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 16:52:02.399475 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":404,"threads_started":1}
I20260812 16:52:02.404800 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:52:02.408319 15317 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } }
I20260812 16:52:02.408705 15317 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:02.408831 15317 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff77c2ed5b4810aab0d814e16410a3, State: Initialized, Role: FOLLOWER
I20260812 16:52:02.409521 15317 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } }
I20260812 16:52:02.409768 15317 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:02.409902 15317 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:02.410085 15317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:02.412307 15317 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } }
I20260812 16:52:02.412823 15317 leader_election.cc:302] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1ff77c2ed5b4810aab0d814e16410a3; no voters:
I20260812 16:52:02.413575 15317 leader_election.cc:288] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:02.413748 15324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:02.415985 15317 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:02.417512 15324 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [term 1 LEADER]: Becoming Leader. State: Replica: f1ff77c2ed5b4810aab0d814e16410a3, State: Running, Role: LEADER
I20260812 16:52:02.418347 15324 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } }
I20260812 16:52:02.421229 15327 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1ff77c2ed5b4810aab0d814e16410a3. Latest consensus state: current_term: 1 leader_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } } }
I20260812 16:52:02.421614 15327 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:02.422197 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1ff77c2ed5b4810aab0d814e16410a3" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34961 } } }
I20260812 16:52:02.422552 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:02.430197 15337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:02.435928 15337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:02.460660 15337 catalog_manager.cc:1347] Generated new cluster ID: be652469c1f2410ca65eaecc973926a7
I20260812 16:52:02.460816 15337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:02.481072 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:02.472987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.8.195.190:34961)
0812 16:52:02.473270 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:02.473280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:02.473894 (+ 614us) server_negotiation.cc:407] Connection header received
0812 16:52:02.474111 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:02.474117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:02.474199 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:02.474328 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:02.475442 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:02.476198 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:02.477648 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:02.478131 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:02.478308 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:02.479430 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:02.479436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:02.479441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:02.480175 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:02.480222 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:02.480249 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:02.480383 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:02.480524 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 16:52:02.480639 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 16:52:02.485327 15337 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:02.486726 15337 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:02.499845 15337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1ff77c2ed5b4810aab0d814e16410a3: Generated new TSK 0
I20260812 16:52:02.500720 15337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:52:02.511695 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15168
I20260812 16:52:02.512984 15308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:02.606009 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15168
2026-08-12T16:52:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3411 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:02 dist-test-slave-z8x9 krb5kdc[15381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:02 dist-test-slave-z8x9 krb5kdc[15381](info): set up 2 sockets
Aug 12 16:52:02 dist-test-slave-z8x9 krb5kdc[15381](info): commencing operation
krb5kdc: starting...
W20260812 16:52:04.012246 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:04 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:04.253156 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:43815
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:43815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:04.587636 15412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:04.587920 15412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:04.588008 15412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:04.594981 15412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:04.595155 15412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:04.595237 15412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:04.595296 15412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:04.595341 15412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:04.604990 15412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:43815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:43815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:04.606374 15412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:04.607640 15412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:04.619650 15417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:04.619565 15419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:04.619616 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:04.620446 15412 server_base.cc:1034] running on GCE node
I20260812 16:52:04.621377 15412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:04.622802 15412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:04.624043 15412 hybrid_clock.cc:648] HybridClock initialized: now 1786553524624000 us; error 54 us; skew 500 ppm
I20260812 16:52:04.624402 15412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:04 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:04.627692 15412 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:04.629217 15412 webserver.cc:460] Webserver started at http://127.8.195.190:36621/ using document root <none> and password file <none>
I20260812 16:52:04.629923 15412 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:04.630076 15412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:04.630396 15412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:04.633424 15412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "da61b622ff3547c089b4008c46f11420"
format_stamp: "Formatted at 2026-08-12 16:52:04 on dist-test-slave-z8x9"
server_key: "e005cd5e611105e438fb4b3f93ed62bc"
server_key_iv: "7d6cbc35fe699806ae8ec207c3eefbb1"
server_key_version: "encryptionkey@0"
I20260812 16:52:04.634218 15412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "da61b622ff3547c089b4008c46f11420"
format_stamp: "Formatted at 2026-08-12 16:52:04 on dist-test-slave-z8x9"
server_key: "e005cd5e611105e438fb4b3f93ed62bc"
server_key_iv: "7d6cbc35fe699806ae8ec207c3eefbb1"
server_key_version: "encryptionkey@0"
I20260812 16:52:04.639432 15412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260812 16:52:04.642868 15440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:04.644176 15412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:52:04.644366 15412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "da61b622ff3547c089b4008c46f11420"
format_stamp: "Formatted at 2026-08-12 16:52:04 on dist-test-slave-z8x9"
server_key: "e005cd5e611105e438fb4b3f93ed62bc"
server_key_iv: "7d6cbc35fe699806ae8ec207c3eefbb1"
server_key_version: "encryptionkey@0"
I20260812 16:52:04.644572 15412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:04.667279 15412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:04.672399 15412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:04.672713 15412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:04.690949 15412 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:43815
I20260812 16:52:04.691030 15545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:43815 every 8 connection(s)
I20260812 16:52:04.692370 15412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:04.696035 15549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:04.697223 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15412
I20260812 16:52:04.697455 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:04.698149 8974 external_mini_cluster.cc:1428] Setting key ca2fe7744b3b2fce12d16115b9c74896
I20260812 16:52:04.711150 15549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420: Bootstrap starting.
Aug 12 16:52:04 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:04.715049 15549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:04.716235 15549 log.cc:824] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:04.719648 15549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420: No bootstrap required, opened a new log
I20260812 16:52:04.723635 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:04.700218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.8.195.190:43815)
0812 16:52:04.701071 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:04.701087 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:04.701136 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:52:04.703318 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:04.703343 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:04.703757 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:04.704102 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:04.705995 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:04.707348 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:04.709239 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:04.709913 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:04.710042 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:04.714886 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:04.714896 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:04.714910 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:04.714954 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:04.718820 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:04.719562 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:04.719570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:04.719579 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:04.719860 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:04.720256 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:04.720261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:04.720264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:04.720975 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:04.721331 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:04.721708 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 16:52:04.722201 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":408,"threads_started":1}
I20260812 16:52:04.729317 15549 raft_consensus.cc:348] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } }
I20260812 16:52:04.729696 15549 raft_consensus.cc:374] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:04.729775 15549 raft_consensus.cc:729] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da61b622ff3547c089b4008c46f11420, State: Initialized, Role: FOLLOWER
I20260812 16:52:04.730342 15549 consensus_queue.cc:260] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } }
I20260812 16:52:04.730593 15549 raft_consensus.cc:388] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:04.730775 15549 raft_consensus.cc:482] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:04.730979 15549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:04.733319 15549 raft_consensus.cc:504] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } }
I20260812 16:52:04.733789 15549 leader_election.cc:302] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da61b622ff3547c089b4008c46f11420; no voters:
I20260812 16:52:04.734519 15549 leader_election.cc:288] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:04.734658 15555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:04.735410 15555 raft_consensus.cc:686] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [term 1 LEADER]: Becoming Leader. State: Replica: da61b622ff3547c089b4008c46f11420, State: Running, Role: LEADER
I20260812 16:52:04.736024 15555 consensus_queue.cc:237] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } }
I20260812 16:52:04.736727 15549 sys_catalog.cc:564] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:04.738718 15568 sys_catalog.cc:455] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da61b622ff3547c089b4008c46f11420. Latest consensus state: current_term: 1 leader_uuid: "da61b622ff3547c089b4008c46f11420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } } }
I20260812 16:52:04.739096 15568 sys_catalog.cc:458] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:04.744457 15558 sys_catalog.cc:455] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da61b622ff3547c089b4008c46f11420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da61b622ff3547c089b4008c46f11420" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43815 } } }
I20260812 16:52:04.744866 15558 sys_catalog.cc:458] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:04.753670 15573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:04.757992 15573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:04.765798 15573 catalog_manager.cc:1347] Generated new cluster ID: a3718992a7d8410ea599d3618ac2d7e4
I20260812 16:52:04.765931 15573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:04.781499 15573 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:04.782981 15573 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:04.790215 15573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da61b622ff3547c089b4008c46f11420: Generated new TSK 0
I20260812 16:52:04.791050 15573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:52:04.852980 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:05.199656 15596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:05.199976 15596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:05.200142 15596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:05.207607 15596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:05.207737 15596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:05.207926 15596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:05.218251 15596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:05.219655 15596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:05.221036 15596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:05.234905 15601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:05.234824 15602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:05.234859 15608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:05.235262 15596 server_base.cc:1034] running on GCE node
I20260812 16:52:05.236239 15596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:05.237430 15596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:05.238651 15596 hybrid_clock.cc:648] HybridClock initialized: now 1786553525238591 us; error 48 us; skew 500 ppm
I20260812 16:52:05.239005 15596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:05 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553525, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:05.246865 15596 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:52:05.248651 15596 webserver.cc:460] Webserver started at http://127.8.195.129:40137/ using document root <none> and password file <none>
I20260812 16:52:05.249504 15596 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:05.249624 15596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:05.249913 15596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:05.252995 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "1198b9fa266f41e99c93c56a90371f3b"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "a6bcc42d133c161515698b5c2b427d41"
server_key_iv: "f7f2ca0245187b30468793cd74f11902"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.253916 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "1198b9fa266f41e99c93c56a90371f3b"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "a6bcc42d133c161515698b5c2b427d41"
server_key_iv: "f7f2ca0245187b30468793cd74f11902"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.259533 15596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:05.263180 15618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.264575 15596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:05.264778 15596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "1198b9fa266f41e99c93c56a90371f3b"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "a6bcc42d133c161515698b5c2b427d41"
server_key_iv: "f7f2ca0245187b30468793cd74f11902"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.265002 15596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:05.288374 15596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:05.293457 15596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:05.293766 15596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:05.294776 15596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:05.296164 15596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:05.296286 15596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.296437 15596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:05.296512 15596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.331773 15596 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:46545
I20260812 16:52:05.331867 15834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:46545 every 8 connection(s)
I20260812 16:52:05.333108 15596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:05.339386 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15596
I20260812 16:52:05.339622 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:05.340516 8974 external_mini_cluster.cc:1428] Setting key 8c96ee0739163c3f3f43a1760168576b
Aug 12 16:52:05 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553525, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:05.356958 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:05.335798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:52031 (local address 127.8.195.190:43815)
0812 16:52:05.336260 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:05.336265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:05.337399 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 16:52:05.340033 (+ 2634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:05.340038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:05.340110 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:05.340218 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:05.342274 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.343195 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.345861 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.346262 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.346373 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:05.350791 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:05.350796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:05.350799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:05.350832 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:05.353634 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:05.354433 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:05.354438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:05.354440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:05.354564 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:05.355036 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:05.355048 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:05.355050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:05.355336 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:05.355499 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:05.356325 (+ 826us) server_negotiation.cc:299] Negotiation successful
0812 16:52:05.356565 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260812 16:52:05.358023 15841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:05.336346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:43815 (local address 127.8.195.129:52031)
0812 16:52:05.337241 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:05.337293 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:05.339690 (+ 2397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:05.340514 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:05.340526 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:05.341344 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:05.342106 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:05.342124 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.343358 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.343363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:05.345672 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:05.345690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.346508 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.346518 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:05.346877 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:05.347673 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:05.347711 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:05.350536 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:05.353788 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:05.353798 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:05.353817 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:05.354291 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:05.354690 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:05.354696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:05.354700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:05.354914 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:05.355472 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:05.355481 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:05.355916 (+ 435us) client_negotiation.cc:769] Sending connection context
0812 16:52:05.356199 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 16:52:05.356785 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":304,"threads_started":1}
I20260812 16:52:05.359724 15835 heartbeater.cc:347] Connected to a master server at 127.8.195.190:43815
I20260812 16:52:05.360065 15835 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:05.360839 15835 heartbeater.cc:511] Master 127.8.195.190:43815 requested a full tablet report, sending...
I20260812 16:52:05.363461 15472 ts_manager.cc:194] Registered new tserver with Master: 1198b9fa266f41e99c93c56a90371f3b (127.8.195.129:46545)
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
I20260812 16:52:05.365458 15472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:52031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:52:05.396616 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:05.735357 15847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:05.735641 15847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:05.735735 15847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:05.743225 15847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:05.743371 15847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:05.743564 15847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:05.754040 15847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:05.755363 15847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:05.756742 15847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:05.769898 15859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:05.769971 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:05.770252 15856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:05.770818 15847 server_base.cc:1034] running on GCE node
I20260812 16:52:05.771441 15847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:05.772404 15847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:05.773631 15847 hybrid_clock.cc:648] HybridClock initialized: now 1786553525773577 us; error 34 us; skew 500 ppm
I20260812 16:52:05.774006 15847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:05 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553525, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:05.777967 15847 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:52:05.779546 15847 webserver.cc:460] Webserver started at http://127.8.195.130:42191/ using document root <none> and password file <none>
I20260812 16:52:05.780321 15847 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:05.780438 15847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:05.780735 15847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:05.784085 15847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "1bc244a8859c46349faf6302d98bd0d6"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "af30a9549a1efb552e2cbd2f97fb162e"
server_key_iv: "fb4ab7d391f001da01e29e94ed727539"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.784924 15847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "1bc244a8859c46349faf6302d98bd0d6"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "af30a9549a1efb552e2cbd2f97fb162e"
server_key_iv: "fb4ab7d391f001da01e29e94ed727539"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.790664 15847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:52:05.794313 15875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.795673 15847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:05.795866 15847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "1bc244a8859c46349faf6302d98bd0d6"
format_stamp: "Formatted at 2026-08-12 16:52:05 on dist-test-slave-z8x9"
server_key: "af30a9549a1efb552e2cbd2f97fb162e"
server_key_iv: "fb4ab7d391f001da01e29e94ed727539"
server_key_version: "encryptionkey@0"
I20260812 16:52:05.796085 15847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:05.810153 15847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:05.814977 15847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:05.815332 15847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:05.816457 15847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:05.817979 15847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:05.818099 15847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.818267 15847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:05.818356 15847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:05.854084 15847 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:32853
I20260812 16:52:05.854120 16102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:32853 every 8 connection(s)
I20260812 16:52:05.855491 15847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:05.860917 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15847
I20260812 16:52:05.861135 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:05.861665 8974 external_mini_cluster.cc:1428] Setting key 851a837eb034d17f04069705bdd13c04
Aug 12 16:52:05 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553525, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:05.878813 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:05.858395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:51651 (local address 127.8.195.190:43815)
0812 16:52:05.858925 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:05.858930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:05.859618 (+ 688us) server_negotiation.cc:407] Connection header received
0812 16:52:05.861769 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:05.861778 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:05.861870 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:05.862001 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:05.863854 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.864652 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.866445 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.866925 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.867081 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:05.871419 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:05.871424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:05.871427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:05.871460 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:05.875212 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:05.875996 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:05.876005 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:05.876011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:05.876203 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:05.876625 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:05.876633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:05.876638 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:05.877048 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:05.877248 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:05.877967 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 16:52:05.878267 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":284,"threads_started":1}
I20260812 16:52:05.879395 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:05.858679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:43815 (local address 127.8.195.130:51651)
0812 16:52:05.859442 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:05.859493 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:05.861449 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:05.862219 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:05.862227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:05.862966 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:05.863665 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:05.863695 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.864825 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.864830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:05.866274 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:05.866288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:05.867181 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:05.867189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:05.867445 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:05.868226 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:05.868251 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:05.871145 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:05.875374 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:05.875381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:05.875394 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:05.875847 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:05.876317 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:05.876321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:05.876323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:05.876507 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:05.877242 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:05.877249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:05.877571 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 16:52:05.877811 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 16:52:05.878211 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":337,"threads_started":1}
I20260812 16:52:05.880929 16104 heartbeater.cc:347] Connected to a master server at 127.8.195.190:43815
I20260812 16:52:05.881299 16104 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:05.882053 16104 heartbeater.cc:511] Master 127.8.195.190:43815 requested a full tablet report, sending...
I20260812 16:52:05.883618 15472 ts_manager.cc:194] Registered new tserver with Master: 1bc244a8859c46349faf6302d98bd0d6 (127.8.195.130:32853)
I20260812 16:52:05.884451 15472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:51651
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:52:05.916500 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:06.249676 16118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:06.249990 16118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:06.250165 16118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:06.257189 16118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:06.257349 16118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:06.257539 16118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:06.267237 16118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:06.268639 16118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:06.270043 16118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:06.283284 16128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:06.283378 16135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:06.283375 16125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:06.284428 16118 server_base.cc:1034] running on GCE node
I20260812 16:52:06.285074 16118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:06.286078 16118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:06.287300 16118 hybrid_clock.cc:648] HybridClock initialized: now 1786553526287240 us; error 43 us; skew 500 ppm
I20260812 16:52:06.287624 16118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:06.290992 16118 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:52:06.292559 16118 webserver.cc:460] Webserver started at http://127.8.195.131:43841/ using document root <none> and password file <none>
I20260812 16:52:06.293347 16118 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:06.293524 16118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:06.293864 16118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:06.296741 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "86891661226f4ddb8be7699c3918d25d"
format_stamp: "Formatted at 2026-08-12 16:52:06 on dist-test-slave-z8x9"
server_key: "5c6c0e184f7be60589ebb8738d3f01a1"
server_key_iv: "78c112cbbb744c606b0e43bd3e1c53ca"
server_key_version: "encryptionkey@0"
I20260812 16:52:06.297595 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "86891661226f4ddb8be7699c3918d25d"
format_stamp: "Formatted at 2026-08-12 16:52:06 on dist-test-slave-z8x9"
server_key: "5c6c0e184f7be60589ebb8738d3f01a1"
server_key_iv: "78c112cbbb744c606b0e43bd3e1c53ca"
server_key_version: "encryptionkey@0"
I20260812 16:52:06.302719 16118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:06.306087 16150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:06.307374 16118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:06.307570 16118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "86891661226f4ddb8be7699c3918d25d"
format_stamp: "Formatted at 2026-08-12 16:52:06 on dist-test-slave-z8x9"
server_key: "5c6c0e184f7be60589ebb8738d3f01a1"
server_key_iv: "78c112cbbb744c606b0e43bd3e1c53ca"
server_key_version: "encryptionkey@0"
I20260812 16:52:06.307781 16118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:06.336529 16118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:06.341045 16118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:06.341421 16118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:06.342440 16118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:06.343865 16118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:06.343942 16118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:06.344033 16118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:06.344085 16118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:06.368836 15835 heartbeater.cc:503] Master 127.8.195.190:43815 was elected leader, sending a full tablet report...
I20260812 16:52:06.377775 16118 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:33847
I20260812 16:52:06.377812 16379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:33847 every 8 connection(s)
I20260812 16:52:06.379221 16118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:06.389037 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16118
I20260812 16:52:06.389307 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:06.389789 8974 external_mini_cluster.cc:1428] Setting key 764624326551cc2fa3c19259a7152b8b
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:06.400208 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.382111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:40697 (local address 127.8.195.190:43815)
0812 16:52:06.382621 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:06.382626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:06.383386 (+ 760us) server_negotiation.cc:407] Connection header received
0812 16:52:06.385231 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:06.385237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:06.385349 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:06.385477 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:06.387015 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.387820 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.389508 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.389940 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.390089 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.394243 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:06.394248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:06.394252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:06.394285 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:06.397050 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.397956 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.397960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.397963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.398095 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.398559 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.398563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.398565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.398850 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:06.398980 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:06.399721 (+ 741us) server_negotiation.cc:299] Negotiation successful
0812 16:52:06.399920 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":283,"threads_started":1}
I20260812 16:52:06.401403 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.382482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:43815 (local address 127.8.195.131:40697)
0812 16:52:06.383215 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:06.383257 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:06.384924 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:06.385721 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:06.385728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:06.386263 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:06.386839 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:06.386856 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.387975 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.387985 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:06.389332 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:06.389345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.390189 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.390199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:06.390465 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.391246 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:06.391285 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:06.394051 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:06.397317 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:06.397324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:06.397338 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:06.397831 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:06.398207 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:06.398210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:06.398212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:06.398417 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:06.398997 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:06.399004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:06.399352 (+ 348us) client_negotiation.cc:769] Sending connection context
0812 16:52:06.399586 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 16:52:06.400107 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":325,"threads_started":1}
I20260812 16:52:06.402942 16380 heartbeater.cc:347] Connected to a master server at 127.8.195.190:43815
I20260812 16:52:06.403267 16380 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:06.404138 16380 heartbeater.cc:511] Master 127.8.195.190:43815 requested a full tablet report, sending...
I20260812 16:52:06.405916 15467 ts_manager.cc:194] Registered new tserver with Master: 86891661226f4ddb8be7699c3918d25d (127.8.195.131:33847)
I20260812 16:52:06.406888 15467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:40697
I20260812 16:52:06.418393 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:06.475970 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.461241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.8.195.190:43815)
0812 16:52:06.461476 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:06.461486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:06.461935 (+ 449us) server_negotiation.cc:407] Connection header received
0812 16:52:06.462090 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:06.462094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:06.462161 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:06.462250 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:06.463710 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.464450 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.465908 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.466331 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.466431 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.470578 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:06.470583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:06.470586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:06.470626 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:06.473267 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.473986 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.473990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.473992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.474118 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.474529 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.474532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.474535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.474893 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:06.475087 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:06.475404 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 16:52:06.475590 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 16:52:06.476714 15467 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51306
I20260812 16:52:06.478704 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15596
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:06.495540 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.479828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.8.195.129:46545)
0812 16:52:06.480518 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:06.480526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:06.480552 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:52:06.480611 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:06.480619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:06.480835 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:06.481015 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:06.482346 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.483373 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.484624 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.485332 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.485489 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.489826 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:06.489838 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:06.489852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:06.489887 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:06.492656 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.493375 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.493391 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.493393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.493524 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.493928 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.493932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.493934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.494290 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:06.494619 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:06.494915 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 16:52:06.495185 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":329,"threads_started":1}
W20260812 16:52:06.496810 15819 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50850
W20260812 16:52:06.498064 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15596: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:52:06.498337 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15596
I20260812 16:52:06.530007 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15847
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:52:06.547024 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.531051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.8.195.130:32853)
0812 16:52:06.531693 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:06.531701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:06.531730 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:52:06.531793 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:06.531800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:06.532028 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:06.532167 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:06.533650 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.534708 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.535994 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.536564 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.536700 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.541365 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:06.541379 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:06.541395 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:06.541432 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:06.544337 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.545024 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.545033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.545035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.545258 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.545640 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.545645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.545647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.546029 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:06.546348 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:06.546508 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:52:06.546728 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":348,"threads_started":1}
W20260812 16:52:06.548563 16092 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46464
W20260812 16:52:06.550002 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15847: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:52:06.550194 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15847
I20260812 16:52:06.578296 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16118
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[15381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553526, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:52:06.595053 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:06.579405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.8.195.131:33847)
0812 16:52:06.580031 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:06.580038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:06.580067 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:52:06.580128 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:06.580134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:06.580347 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:06.580473 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:06.581899 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.582950 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.584179 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:06.584741 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:06.584845 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:06.589393 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:06.589406 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:06.589421 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:06.589458 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:06.592423 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.593100 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.593108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.593111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.593291 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:06.593669 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:06.593672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:06.593674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:06.594031 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:06.594271 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:06.594533 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 16:52:06.594741 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":306,"threads_started":1}
W20260812 16:52:06.596402 16374 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37520
W20260812 16:52:06.597710 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16118: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:52:06.597895 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16118
I20260812 16:52:06.621061 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15412
I20260812 16:52:06.622361 15537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:06.730803 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15412
2026-08-12T16:52:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4125 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[16419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[16419](info): set up 2 sockets
Aug 12 16:52:06 dist-test-slave-z8x9 krb5kdc[16419](info): commencing operation
krb5kdc: starting...
W20260812 16:52:08.123980 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:08 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:08.364979 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34173
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:08.700830 16440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:08.701225 16440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:08.701401 16440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:08.708652 16440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:08.708799 16440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:08.708913 16440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:08.709020 16440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:08.709110 16440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:08.719038 16440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:08.720444 16440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:08.721796 16440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:08.734416 16447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:08.734503 16455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:08.734480 16452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:08.735401 16440 server_base.cc:1034] running on GCE node
I20260812 16:52:08.736208 16440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:08.737746 16440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:08.738991 16440 hybrid_clock.cc:648] HybridClock initialized: now 1786553528738951 us; error 46 us; skew 500 ppm
I20260812 16:52:08.739354 16440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:08 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553528, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:08.742643 16440 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:08.744241 16440 webserver.cc:460] Webserver started at http://127.8.195.190:37451/ using document root <none> and password file <none>
I20260812 16:52:08.744966 16440 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:08.745124 16440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:08.745509 16440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:08.748539 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "26385eefb0a54734a81acf61b4bc4b97"
format_stamp: "Formatted at 2026-08-12 16:52:08 on dist-test-slave-z8x9"
server_key: "9514860a38008bddd74a1d5da139a5b6"
server_key_iv: "43ab51dbbea2201b594791bf005e78e8"
server_key_version: "encryptionkey@0"
I20260812 16:52:08.749437 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "26385eefb0a54734a81acf61b4bc4b97"
format_stamp: "Formatted at 2026-08-12 16:52:08 on dist-test-slave-z8x9"
server_key: "9514860a38008bddd74a1d5da139a5b6"
server_key_iv: "43ab51dbbea2201b594791bf005e78e8"
server_key_version: "encryptionkey@0"
I20260812 16:52:08.754835 16440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:08.758268 16465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:08.759528 16440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:08.759727 16440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "26385eefb0a54734a81acf61b4bc4b97"
format_stamp: "Formatted at 2026-08-12 16:52:08 on dist-test-slave-z8x9"
server_key: "9514860a38008bddd74a1d5da139a5b6"
server_key_iv: "43ab51dbbea2201b594791bf005e78e8"
server_key_version: "encryptionkey@0"
I20260812 16:52:08.759932 16440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:08.781757 16440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:08.786051 16440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:08.786373 16440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:08.805868 16440 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34173
I20260812 16:52:08.805907 16567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34173 every 8 connection(s)
I20260812 16:52:08.807243 16440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:08.807775 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16440
I20260812 16:52:08.808022 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:08.808861 8974 external_mini_cluster.cc:1428] Setting key bf3eac20122aa1f7fd6037778b138f9c
I20260812 16:52:08.811816 16570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:52:08 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:08.824947 16570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97: Bootstrap starting.
I20260812 16:52:08.828537 16570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:08.829718 16570 log.cc:824] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:08.833065 16570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97: No bootstrap required, opened a new log
I20260812 16:52:08.833662 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:08.810996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.8.195.190:34173)
0812 16:52:08.811831 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:08.811847 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:08.811902 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:52:08.814279 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:08.814319 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:08.814805 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:08.815235 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:08.816695 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:08.818799 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:08.820129 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:08.821198 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:08.821504 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:08.825932 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:08.825940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:08.825955 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:08.825998 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:08.829336 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:08.830106 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:08.830120 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:08.830126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:08.830408 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:08.830823 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:08.830833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:08.830837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:08.831475 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:08.831697 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:08.832165 (+ 468us) server_negotiation.cc:299] Negotiation successful
0812 16:52:08.832530 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":345,"threads_started":1}
I20260812 16:52:08.841467 16570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } }
I20260812 16:52:08.841861 16570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:08.841971 16570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26385eefb0a54734a81acf61b4bc4b97, State: Initialized, Role: FOLLOWER
I20260812 16:52:08.842640 16570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } }
I20260812 16:52:08.842938 16570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:08.843098 16570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:08.843288 16570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:08.845501 16570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } }
I20260812 16:52:08.846005 16570 leader_election.cc:302] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26385eefb0a54734a81acf61b4bc4b97; no voters:
I20260812 16:52:08.846572 16570 leader_election.cc:288] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:08.846702 16581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:08.847457 16581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [term 1 LEADER]: Becoming Leader. State: Replica: 26385eefb0a54734a81acf61b4bc4b97, State: Running, Role: LEADER
I20260812 16:52:08.848012 16581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } }
I20260812 16:52:08.848718 16570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:08.850582 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26385eefb0a54734a81acf61b4bc4b97. Latest consensus state: current_term: 1 leader_uuid: "26385eefb0a54734a81acf61b4bc4b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } } }
I20260812 16:52:08.851040 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:08.851441 16586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26385eefb0a54734a81acf61b4bc4b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26385eefb0a54734a81acf61b4bc4b97" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34173 } } }
I20260812 16:52:08.851790 16586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:08.852284 16596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:08.858335 16596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:08.867154 16596 catalog_manager.cc:1347] Generated new cluster ID: 442f241621b24e038f0baf34ed873b0c
I20260812 16:52:08.867268 16596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:08.886670 16596 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:08.888581 16596 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:08.901841 16596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26385eefb0a54734a81acf61b4bc4b97: Generated new TSK 0
I20260812 16:52:08.902705 16596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:52:08.965610 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:09.304260 16613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:09.304551 16613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:09.304714 16613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:09.311921 16613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:09.312024 16613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:09.312229 16613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:09.322027 16613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:09.323264 16613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:09.324608 16613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:09.338181 16628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:09.338181 16629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:09.338599 16631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:09.339334 16613 server_base.cc:1034] running on GCE node
I20260812 16:52:09.339910 16613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:09.340821 16613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:09.342022 16613 hybrid_clock.cc:648] HybridClock initialized: now 1786553529341981 us; error 27 us; skew 500 ppm
I20260812 16:52:09.342298 16613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:09 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:09.345592 16613 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:52:09.346995 16613 webserver.cc:460] Webserver started at http://127.8.195.129:35597/ using document root <none> and password file <none>
I20260812 16:52:09.347692 16613 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:09.347842 16613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:09.348166 16613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:09.351328 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "562c244a122c49edb22b1779ed9e07eb"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "98be7a9ce2ae2513f9b65edacade99f6"
server_key_iv: "e25cac4ad4537bc71ce40e6542d739df"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.352253 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "562c244a122c49edb22b1779ed9e07eb"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "98be7a9ce2ae2513f9b65edacade99f6"
server_key_iv: "e25cac4ad4537bc71ce40e6542d739df"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.357957 16613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 16:52:09.361429 16642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.362713 16613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:09.362896 16613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "562c244a122c49edb22b1779ed9e07eb"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "98be7a9ce2ae2513f9b65edacade99f6"
server_key_iv: "e25cac4ad4537bc71ce40e6542d739df"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.363114 16613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:09.380173 16613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:09.384673 16613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:09.384927 16613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:09.385869 16613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:09.387159 16613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:09.387235 16613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.387331 16613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:09.387377 16613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.421355 16613 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:45159
I20260812 16:52:09.421397 16848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:45159 every 8 connection(s)
I20260812 16:52:09.422663 16613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:09.429981 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16613
I20260812 16:52:09.430219 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:09.430883 8974 external_mini_cluster.cc:1428] Setting key b29450b6c8840f39d39c74f0e0f4b3dc
Aug 12 16:52:09 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:09.442588 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:09.425158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36211 (local address 127.8.195.190:34173)
0812 16:52:09.425700 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:09.425707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:09.426642 (+ 935us) server_negotiation.cc:407] Connection header received
0812 16:52:09.428427 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:09.428432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:09.428508 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:09.428606 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:09.430273 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.431099 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.432630 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.433040 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.433203 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:09.436718 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:09.436723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:09.436727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:09.436761 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:09.439440 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:09.440174 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:09.440186 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:09.440189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:09.440312 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:09.440774 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:09.440778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:09.440780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:09.441035 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:09.441221 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:09.441986 (+ 765us) server_negotiation.cc:299] Negotiation successful
0812 16:52:09.442224 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":228,"threads_started":1}
I20260812 16:52:09.443555 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:09.425780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34173 (local address 127.8.195.129:36211)
0812 16:52:09.426482 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:09.426520 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:09.428120 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:09.428857 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:09.428865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:09.429474 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:09.430077 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:09.430105 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.431280 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.431284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:09.432483 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:09.432496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.433302 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.433312 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:09.433542 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:09.434288 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:09.434316 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:09.436471 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:09.439586 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:09.439596 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:09.439611 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:09.440040 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:09.440437 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:09.440442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:09.440446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:09.440636 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:09.441184 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:09.441193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:09.441582 (+ 389us) client_negotiation.cc:769] Sending connection context
0812 16:52:09.441864 (+ 282us) client_negotiation.cc:240] Negotiation successful
0812 16:52:09.442399 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":265,"threads_started":1}
I20260812 16:52:09.445230 16850 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34173
I20260812 16:52:09.445529 16850 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:09.446252 16850 heartbeater.cc:511] Master 127.8.195.190:34173 requested a full tablet report, sending...
I20260812 16:52:09.448370 16490 ts_manager.cc:194] Registered new tserver with Master: 562c244a122c49edb22b1779ed9e07eb (127.8.195.129:45159)
I20260812 16:52:09.450073 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:36211
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:52:09.484957 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:09.852519 16858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:09.852850 16858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:09.853024 16858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:09.860496 16858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:09.860637 16858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:09.860841 16858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:09.871031 16858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:09.872321 16858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:09.873692 16858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:09.887422 16869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:09.887564 16863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:09.887696 16874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:09.889075 16858 server_base.cc:1034] running on GCE node
I20260812 16:52:09.889912 16858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:09.891166 16858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:09.892438 16858 hybrid_clock.cc:648] HybridClock initialized: now 1786553529892390 us; error 46 us; skew 500 ppm
I20260812 16:52:09.892887 16858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:09 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:09.897828 16858 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:52:09.899446 16858 webserver.cc:460] Webserver started at http://127.8.195.130:43091/ using document root <none> and password file <none>
I20260812 16:52:09.900127 16858 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:09.900264 16858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:09.900595 16858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:09.903633 16858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "ad7b9cc9d44148918a3b5e08a2164ed0"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "8d301124c91794201b2625745bc82b12"
server_key_iv: "ce604562c104b94a2f65589c8dbea114"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.904479 16858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "ad7b9cc9d44148918a3b5e08a2164ed0"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "8d301124c91794201b2625745bc82b12"
server_key_iv: "ce604562c104b94a2f65589c8dbea114"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.910377 16858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 16:52:09.914175 16883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.915728 16858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:09.915920 16858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "ad7b9cc9d44148918a3b5e08a2164ed0"
format_stamp: "Formatted at 2026-08-12 16:52:09 on dist-test-slave-z8x9"
server_key: "8d301124c91794201b2625745bc82b12"
server_key_iv: "ce604562c104b94a2f65589c8dbea114"
server_key_version: "encryptionkey@0"
I20260812 16:52:09.916128 16858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:09.931859 16858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:09.936314 16858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:09.936604 16858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:09.937767 16858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:09.939148 16858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:09.939260 16858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.939414 16858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:09.939507 16858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:09.974874 16858 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:38383
I20260812 16:52:09.974910 17118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:38383 every 8 connection(s)
I20260812 16:52:09.976271 16858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:09.978193 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16858
I20260812 16:52:09.978412 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:09.978888 8974 external_mini_cluster.cc:1428] Setting key a71a3b0ee33dbe0a310c0f5e71e20138
Aug 12 16:52:09 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:10.001632 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:09.979153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:57063 (local address 127.8.195.190:34173)
0812 16:52:09.979656 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:09.979663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:09.980633 (+ 970us) server_negotiation.cc:407] Connection header received
0812 16:52:09.983419 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:09.983423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:09.983486 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:09.983580 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:09.985738 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.986497 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.988982 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.989491 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.989591 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:09.993908 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:09.993918 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:09.993926 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:09.993976 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:09.997934 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:09.998751 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:09.998759 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:09.998765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:09.998935 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:09.999414 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:09.999422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:09.999428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:09.999828 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.000015 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.000772 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.001055 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260812 16:52:10.002417 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:09.979736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34173 (local address 127.8.195.130:57063)
0812 16:52:09.980478 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:09.980517 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:09.983109 (+ 2592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:09.983888 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:09.983903 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:09.984719 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:09.985558 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:09.985586 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.986663 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.986672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:09.988823 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:09.988841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:09.989723 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:09.989733 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:09.990064 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:09.990922 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:09.990946 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:09.993615 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:09.998099 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:09.998107 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:09.998121 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:09.998592 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:09.999057 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:09.999060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:09.999062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:09.999275 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:09.999995 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:10.000001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:10.000372 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 16:52:10.000617 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 16:52:10.001053 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":317,"threads_started":1}
I20260812 16:52:10.004102 17120 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34173
I20260812 16:52:10.004483 17120 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:10.005527 17120 heartbeater.cc:511] Master 127.8.195.190:34173 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
I20260812 16:52:10.007359 16490 ts_manager.cc:194] Registered new tserver with Master: ad7b9cc9d44148918a3b5e08a2164ed0 (127.8.195.130:38383)
I20260812 16:52:10.008256 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:57063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:52:10.037933 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:10.372812 17129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:10.373106 17129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:10.373273 17129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:10.380414 17129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:10.380525 17129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:10.380640 17129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:10.390374 17129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:10.391633 17129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:10.392927 17129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:10.406659 17142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:10.406718 17146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:10.406750 17143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:10.407840 17129 server_base.cc:1034] running on GCE node
I20260812 16:52:10.408563 17129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:10.409596 17129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:10.410833 17129 hybrid_clock.cc:648] HybridClock initialized: now 1786553530410802 us; error 57 us; skew 500 ppm
I20260812 16:52:10.411161 17129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:10.418924 17129 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:52:10.420596 17129 webserver.cc:460] Webserver started at http://127.8.195.131:35117/ using document root <none> and password file <none>
I20260812 16:52:10.421530 17129 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:10.421689 17129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:10.422070 17129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:10.425987 17129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "2727f7a046874d51a7e71c4fd6a0cf49"
format_stamp: "Formatted at 2026-08-12 16:52:10 on dist-test-slave-z8x9"
server_key: "e9440316f8115776b58e483179a29b78"
server_key_iv: "fa1455139ee4aec34c0f7dfbe4d15b7b"
server_key_version: "encryptionkey@0"
I20260812 16:52:10.426991 17129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "2727f7a046874d51a7e71c4fd6a0cf49"
format_stamp: "Formatted at 2026-08-12 16:52:10 on dist-test-slave-z8x9"
server_key: "e9440316f8115776b58e483179a29b78"
server_key_iv: "fa1455139ee4aec34c0f7dfbe4d15b7b"
server_key_version: "encryptionkey@0"
I20260812 16:52:10.432287 17129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:10.435776 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:10.437115 17129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:10.437343 17129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "2727f7a046874d51a7e71c4fd6a0cf49"
format_stamp: "Formatted at 2026-08-12 16:52:10 on dist-test-slave-z8x9"
server_key: "e9440316f8115776b58e483179a29b78"
server_key_iv: "fa1455139ee4aec34c0f7dfbe4d15b7b"
server_key_version: "encryptionkey@0"
I20260812 16:52:10.437525 17129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:10.453375 16850 heartbeater.cc:503] Master 127.8.195.190:34173 was elected leader, sending a full tablet report...
I20260812 16:52:10.460333 17129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:10.464815 17129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:10.465147 17129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:10.466241 17129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:10.467772 17129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:10.467856 17129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:10.468032 17129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:10.468086 17129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:10.503377 17129 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:40641
I20260812 16:52:10.503422 17396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:40641 every 8 connection(s)
I20260812 16:52:10.504688 17129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:10.511359 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17129
I20260812 16:52:10.511562 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:10.512022 8974 external_mini_cluster.cc:1428] Setting key c36e293cd23b7d5c9fa4621b5388b152
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:10.526230 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.507344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:56559 (local address 127.8.195.190:34173)
0812 16:52:10.507857 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.507864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.508806 (+ 942us) server_negotiation.cc:407] Connection header received
0812 16:52:10.510705 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.510709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.510790 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.510890 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.512649 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.513573 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.515678 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.516083 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.516182 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.520448 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.520453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.520456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.520495 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.523170 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.523860 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.523864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.523866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.523989 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.524384 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.524388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.524390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.524646 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.524767 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.525638 (+ 871us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.525860 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":228,"threads_started":1}
I20260812 16:52:10.527423 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.507916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34173 (local address 127.8.195.131:56559)
0812 16:52:10.508632 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:10.508685 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:10.510444 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:10.511120 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:10.511128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:10.511820 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:10.512436 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:10.512454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.513749 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.513752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:10.515549 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:10.515561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.516311 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.516317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:10.516744 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.517495 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:10.517518 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:10.520248 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:10.523307 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:10.523315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:10.523327 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:10.523756 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:10.524084 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:10.524087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:10.524090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:10.524288 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:10.524800 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:10.524806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:10.525140 (+ 334us) client_negotiation.cc:769] Sending connection context
0812 16:52:10.525506 (+ 366us) client_negotiation.cc:240] Negotiation successful
0812 16:52:10.526008 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":276,"threads_started":1}
I20260812 16:52:10.528865 17399 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34173
I20260812 16:52:10.529134 17399 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:10.529874 17399 heartbeater.cc:511] Master 127.8.195.190:34173 requested a full tablet report, sending...
I20260812 16:52:10.531343 16490 ts_manager.cc:194] Registered new tserver with Master: 2727f7a046874d51a7e71c4fd6a0cf49 (127.8.195.131:40641)
I20260812 16:52:10.532138 16490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:56559
I20260812 16:52:10.540498 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:10.598413 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.583134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.8.195.129:45159)
0812 16:52:10.583935 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.583943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.583968 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:52:10.584026 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.584033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.584247 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.584423 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.585924 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.586917 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.588152 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.588722 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.588881 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.592874 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.592886 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.592900 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.592932 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.595783 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.596492 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.596499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.596502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.596620 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.596942 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.596945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.596947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.597367 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.597599 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.597926 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.598125 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":359,"threads_started":1}
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:10.616346 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.603032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.8.195.190:34173)
0812 16:52:10.603268 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.603279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.603403 (+ 124us) server_negotiation.cc:407] Connection header received
0812 16:52:10.603536 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.603540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.603617 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.603714 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.604893 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.605703 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.607022 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.607459 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.607571 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.611430 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.611436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.611439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.611476 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.614013 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.614612 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.614616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.614618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.614741 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.615052 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.615056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.615057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.615381 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.615579 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.615813 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.615976 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260812 16:52:10.616931 16490 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50746
I20260812 16:52:10.617965 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16613
I20260812 16:52:10.630805 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.618929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.8.195.129:45159)
0812 16:52:10.619132 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.619137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.619244 (+ 107us) server_negotiation.cc:407] Connection header received
0812 16:52:10.619436 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.619442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.619517 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.619623 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.620880 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.621663 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.622891 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.623335 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.623478 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.625719 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.625724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.625728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.625758 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.628221 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.628875 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.628881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.628886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.629015 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.629458 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.629463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.629466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.629707 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.629960 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.630263 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.630494 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:52:10.631511 16835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:10.725116 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16613
I20260812 16:52:10.747424 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16858
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:52:10.763983 17426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.748630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.8.195.130:38383)
0812 16:52:10.749279 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.749287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.749315 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:52:10.749377 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.749384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.749658 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.749803 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.751121 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.752159 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.753628 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.754158 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.754269 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.758551 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.758562 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.758576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.758609 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.761422 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.762042 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.762049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.762052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.762183 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.762517 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.762520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.762522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.762850 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.763074 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.763396 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.763645 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":317,"threads_started":1}
I20260812 16:52:10.765450 17111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:10.847810 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16858
I20260812 16:52:10.873250 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17129
Aug 12 16:52:10 dist-test-slave-z8x9 krb5kdc[16419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:52:10.889861 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:10.874294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41328 (local address 127.8.195.131:40641)
0812 16:52:10.874972 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:10.874979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:10.875010 (+ 31us) server_negotiation.cc:407] Connection header received
0812 16:52:10.875077 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:10.875083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:10.875300 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:10.875430 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:10.876796 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.878038 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.879344 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:10.879899 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:10.879996 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:10.884435 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:10.884446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:10.884460 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:10.884494 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:10.887253 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.887908 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.887916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.887918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.888049 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:10.888361 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:10.888364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:10.888366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:10.888710 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:10.888948 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:10.889253 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 16:52:10.889463 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":354,"threads_started":1}
I20260812 16:52:10.891124 17389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:10.973454 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17129
I20260812 16:52:11.000398 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16440
I20260812 16:52:11.001777 16556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:11.107295 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16440
2026-08-12T16:52:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4374 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:11 dist-test-slave-z8x9 krb5kdc[17437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:11 dist-test-slave-z8x9 krb5kdc[17437](info): set up 2 sockets
Aug 12 16:52:11 dist-test-slave-z8x9 krb5kdc[17437](info): commencing operation
krb5kdc: starting...
W20260812 16:52:12.997217 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.839s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:13 dist-test-slave-z8x9 krb5kdc[17437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:13.289120 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35337
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:13.626024 17458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:13.626324 17458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:13.626521 17458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:13.633690 17458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:13.633800 17458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:13.633908 17458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:13.633993 17458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:13.634076 17458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:13.643967 17458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:13.645329 17458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:13.646688 17458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:13.659045 17468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:13.659188 17467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:13.659188 17472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:13.660333 17458 server_base.cc:1034] running on GCE node
I20260812 16:52:13.661147 17458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:13.662683 17458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:13.663914 17458 hybrid_clock.cc:648] HybridClock initialized: now 1786553533663880 us; error 35 us; skew 500 ppm
I20260812 16:52:13.664317 17458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:13 dist-test-slave-z8x9 krb5kdc[17437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:13.667704 17458 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:13.668887 17458 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:13.669006 17458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:13.669456 17458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:13.672469 17458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "802f99f41d0542f985fae5583fdec55e"
format_stamp: "Formatted at 2026-08-12 16:52:13 on dist-test-slave-z8x9"
server_key: "1798b09d22988867f1d01edb96a74dbd"
server_key_iv: "d9fb8fc95d39ced8f37072f63bf4560b"
server_key_version: "encryptionkey@0"
I20260812 16:52:13.673358 17458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "802f99f41d0542f985fae5583fdec55e"
format_stamp: "Formatted at 2026-08-12 16:52:13 on dist-test-slave-z8x9"
server_key: "1798b09d22988867f1d01edb96a74dbd"
server_key_iv: "d9fb8fc95d39ced8f37072f63bf4560b"
server_key_version: "encryptionkey@0"
I20260812 16:52:13.678444 17458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:13.682245 17490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:13.683676 17458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:13.683889 17458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "802f99f41d0542f985fae5583fdec55e"
format_stamp: "Formatted at 2026-08-12 16:52:13 on dist-test-slave-z8x9"
server_key: "1798b09d22988867f1d01edb96a74dbd"
server_key_iv: "d9fb8fc95d39ced8f37072f63bf4560b"
server_key_version: "encryptionkey@0"
I20260812 16:52:13.684154 17458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:13.703351 17458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:13.707886 17458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:13.708185 17458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:13.727357 17458 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:35337
I20260812 16:52:13.727384 17600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:35337 every 8 connection(s)
I20260812 16:52:13.728397 17458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:13.732412 17603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:13.735153 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17458
I20260812 16:52:13.735477 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:13.736189 8974 external_mini_cluster.cc:1428] Setting key 3db29ab708b2a24ddbfa34f1bc8d6797
I20260812 16:52:13.746613 17603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: Bootstrap starting.
I20260812 16:52:13.751567 17603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:13.753433 17603 log.cc:824] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:52:13 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:13.760048 17603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: No bootstrap required, opened a new log
I20260812 16:52:13.770610 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:13.738455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.8.195.190:35337)
0812 16:52:13.739327 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:13.739341 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:13.739384 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:52:13.741158 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:13.741261 (+ 103us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:13.741812 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:13.742332 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:13.745343 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:13.747241 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:13.749628 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:13.750841 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:13.751019 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:13.758117 (+ 7098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:13.758129 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:13.758150 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:13.758201 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:13.763258 (+ 5057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:13.764430 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:13.764442 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:13.764453 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:13.764714 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:13.765302 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:13.765309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:13.765314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:13.766362 (+ 1048us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:13.768123 (+ 1761us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:13.768570 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 16:52:13.769139 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":399,"threads_started":1}
I20260812 16:52:13.772969 17603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } }
I20260812 16:52:13.773578 17603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:13.773761 17603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802f99f41d0542f985fae5583fdec55e, State: Initialized, Role: FOLLOWER
I20260812 16:52:13.774467 17603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } }
I20260812 16:52:13.774885 17603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:13.775046 17603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:13.775238 17603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:13.779044 17603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } }
I20260812 16:52:13.779743 17603 leader_election.cc:302] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 802f99f41d0542f985fae5583fdec55e; no voters:
I20260812 16:52:13.780484 17603 leader_election.cc:288] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:13.780594 17614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:13.781929 17614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [term 1 LEADER]: Becoming Leader. State: Replica: 802f99f41d0542f985fae5583fdec55e, State: Running, Role: LEADER
I20260812 16:52:13.782537 17614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } }
I20260812 16:52:13.783125 17603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:13.785588 17616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802f99f41d0542f985fae5583fdec55e. Latest consensus state: current_term: 1 leader_uuid: "802f99f41d0542f985fae5583fdec55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } } }
I20260812 16:52:13.786051 17616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:13.785897 17615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "802f99f41d0542f985fae5583fdec55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802f99f41d0542f985fae5583fdec55e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35337 } } }
I20260812 16:52:13.786196 17615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:13.791953 17621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:13.798467 17621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:13.816350 17621 catalog_manager.cc:1347] Generated new cluster ID: aa4297e256a3474eb67ae283ec88068d
I20260812 16:52:13.816538 17621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:13.831053 17621 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:13.833244 17621 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:13.846627 17621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: Generated new TSK 0
I20260812 16:52:13.847631 17621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:52:13.923666 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:14.281059 17659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:14.281436 17659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:14.281617 17659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:14.289476 17659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:14.289606 17659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:14.289794 17659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:14.299686 17659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:14.300971 17659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:14.302371 17659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:14.314965 17671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:14.315088 17672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:14.315495 17678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:14.316242 17659 server_base.cc:1034] running on GCE node
I20260812 16:52:14.316846 17659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:14.317806 17659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:14.319029 17659 hybrid_clock.cc:648] HybridClock initialized: now 1786553534318961 us; error 54 us; skew 500 ppm
I20260812 16:52:14.319384 17659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:14 dist-test-slave-z8x9 krb5kdc[17437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:14.322587 17659 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:52:14.323742 17659 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:14.323879 17659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:14.324195 17659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:14.327162 17659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "6022b68377ff41a5bd397abe169838cc"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "49037fb0c29d33390b50b112fe1842ba"
server_key_iv: "04b30b1520d886576b446de74c260b0c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.328016 17659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "6022b68377ff41a5bd397abe169838cc"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "49037fb0c29d33390b50b112fe1842ba"
server_key_iv: "04b30b1520d886576b446de74c260b0c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.333535 17659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:52:14.336889 17685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.338397 17659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:14.338596 17659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "6022b68377ff41a5bd397abe169838cc"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "49037fb0c29d33390b50b112fe1842ba"
server_key_iv: "04b30b1520d886576b446de74c260b0c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.338853 17659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:14.359917 17659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:14.364362 17659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:14.364671 17659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:14.365883 17659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:14.367443 17659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:14.367544 17659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.367722 17659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:14.367825 17659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.402245 17659 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:37131
I20260812 16:52:14.402302 17898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:37131 every 8 connection(s)
I20260812 16:52:14.403160 17659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:14.403549 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17659
I20260812 16:52:14.403796 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:14.404675 8974 external_mini_cluster.cc:1428] Setting key 6329559ae8b71913217a9b38d4326890
Aug 12 16:52:14 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:14.427925 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:14.406036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49115 (local address 127.8.195.190:35337)
0812 16:52:14.406590 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:14.406596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:14.407831 (+ 1235us) server_negotiation.cc:407] Connection header received
0812 16:52:14.410855 (+ 3024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:14.410860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:14.410936 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:14.411031 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:14.413263 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.414020 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.416518 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.417278 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.417432 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:14.421347 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:14.421353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:14.421356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:14.421390 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:14.424291 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:14.425358 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:14.425362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:14.425364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:14.425490 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:14.426048 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:14.426052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:14.426054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:14.426335 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:14.426444 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:14.427484 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0812 16:52:14.427666 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":322,"threads_started":1}
I20260812 16:52:14.429632 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:14.406750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35337 (local address 127.8.195.129:49115)
0812 16:52:14.407637 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:14.407699 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:14.410537 (+ 2838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:14.411309 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:14.411323 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:14.412147 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:14.413005 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:14.413033 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.414189 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.414198 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:14.416351 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:14.416364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.417490 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.417495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:14.417687 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:14.418419 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:14.418442 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:14.421066 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:14.424461 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:14.424475 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:14.424497 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:14.425244 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:14.425614 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:14.425621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:14.425626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:14.425919 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:14.426482 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:14.426494 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:14.427026 (+ 532us) client_negotiation.cc:769] Sending connection context
0812 16:52:14.427395 (+ 369us) client_negotiation.cc:240] Negotiation successful
0812 16:52:14.428049 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
I20260812 16:52:14.431414 17900 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35337
I20260812 16:52:14.431787 17900 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:14.432473 17900 heartbeater.cc:511] Master 127.8.195.190:35337 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
I20260812 16:52:14.434836 17525 ts_manager.cc:194] Registered new tserver with Master: 6022b68377ff41a5bd397abe169838cc (127.8.195.129:37131)
I20260812 16:52:14.436327 17525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:49115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:52:14.462088 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:14.797111 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:14.797554 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:14.797744 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:14.805238 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:14.805423 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:14.805608 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:14.815771 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:14.817226 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:14.818574 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:14.832304 17920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:14.832309 17919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:14.832311 17923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:14.838413 17909 server_base.cc:1034] running on GCE node
I20260812 16:52:14.839082 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:14.840484 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:14.841697 17909 hybrid_clock.cc:648] HybridClock initialized: now 1786553534841647 us; error 39 us; skew 500 ppm
I20260812 16:52:14.842049 17909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:14 dist-test-slave-z8x9 krb5kdc[17437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:14.845501 17909 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:52:14.846776 17909 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:14.846920 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:14.847223 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:14.850179 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "d065be6f765549a998c1254b2af281b8"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "dd0cf08e05631df5e115bb0d3477a666"
server_key_iv: "c2d524f024685af4bb484f8eca97dd1c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.850978 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "d065be6f765549a998c1254b2af281b8"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "dd0cf08e05631df5e115bb0d3477a666"
server_key_iv: "c2d524f024685af4bb484f8eca97dd1c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.856076 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:52:14.859735 17935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.861416 17909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:52:14.861624 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "d065be6f765549a998c1254b2af281b8"
format_stamp: "Formatted at 2026-08-12 16:52:14 on dist-test-slave-z8x9"
server_key: "dd0cf08e05631df5e115bb0d3477a666"
server_key_iv: "c2d524f024685af4bb484f8eca97dd1c"
server_key_version: "encryptionkey@0"
I20260812 16:52:14.861882 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:14.887576 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:14.891886 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:14.892175 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:14.893278 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:14.894717 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:14.894851 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.894981 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:14.895071 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:14.929033 17909 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:36355
I20260812 16:52:14.929090 18144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:36355 every 8 connection(s)
I20260812 16:52:14.929999 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:14.935282 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17909
I20260812 16:52:14.935518 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:14.936007 8974 external_mini_cluster.cc:1428] Setting key f726daa42f4937dfcb3f91271e5d8c4c
Aug 12 16:52:14 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:14.952970 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:14.932598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:55267 (local address 127.8.195.190:35337)
0812 16:52:14.933059 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:14.933064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:14.934223 (+ 1159us) server_negotiation.cc:407] Connection header received
0812 16:52:14.936294 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:14.936300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:14.936400 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:14.936537 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:14.938738 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.939911 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.941569 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.942330 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.942475 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:14.946459 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:14.946468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:14.946476 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:14.946523 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:14.949488 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:14.950474 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:14.950477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:14.950480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:14.950611 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:14.951107 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:14.951110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:14.951112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:14.951391 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:14.951499 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:14.952501 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0812 16:52:14.952683 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260812 16:52:14.954587 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:14.933228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35337 (local address 127.8.195.130:55267)
0812 16:52:14.934008 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:14.934049 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:14.935950 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:14.936854 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:14.936869 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:14.937741 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:14.938552 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:14.938572 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.940065 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.940069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:14.941426 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:14.941439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:14.942648 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:14.942659 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:14.942874 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:14.943688 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:14.943727 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:14.946174 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:14.949632 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:14.949644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:14.949665 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:14.950350 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:14.950720 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:14.950728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:14.950734 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:14.950992 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:14.951527 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:14.951538 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:14.952018 (+ 480us) client_negotiation.cc:769] Sending connection context
0812 16:52:14.952348 (+ 330us) client_negotiation.cc:240] Negotiation successful
0812 16:52:14.952969 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":332,"threads_started":1}
I20260812 16:52:14.956182 18145 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35337
I20260812 16:52:14.956575 18145 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:14.957367 18145 heartbeater.cc:511] Master 127.8.195.190:35337 requested a full tablet report, sending...
I20260812 16:52:14.959028 17525 ts_manager.cc:194] Registered new tserver with Master: d065be6f765549a998c1254b2af281b8 (127.8.195.130:36355)
I20260812 16:52:14.959878 17525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:55267
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:52:14.996806 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:15.339717 18155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:15.340081 18155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:15.340272 18155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:15.347739 18155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:15.347908 18155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:15.348100 18155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:15.358067 18155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:15.359450 18155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:15.360809 18155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:15.374217 18171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:15.374192 18166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:15.374270 18168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:15.375234 18155 server_base.cc:1034] running on GCE node
I20260812 16:52:15.375941 18155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:15.376942 18155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:15.378185 18155 hybrid_clock.cc:648] HybridClock initialized: now 1786553535378134 us; error 62 us; skew 500 ppm
I20260812 16:52:15.378533 18155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:15 dist-test-slave-z8x9 krb5kdc[17437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553535, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:15.381964 18155 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:52:15.383301 18155 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:15.383404 18155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:15.383678 18155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:15.386895 18155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "1c553275664a44f6a1ebdef169e3f8d0"
format_stamp: "Formatted at 2026-08-12 16:52:15 on dist-test-slave-z8x9"
server_key: "db82b6ea5115acfbb3d07eebb3d6966a"
server_key_iv: "d1a4deeb5972e161db3caeb78f5eea3d"
server_key_version: "encryptionkey@0"
I20260812 16:52:15.387831 18155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "1c553275664a44f6a1ebdef169e3f8d0"
format_stamp: "Formatted at 2026-08-12 16:52:15 on dist-test-slave-z8x9"
server_key: "db82b6ea5115acfbb3d07eebb3d6966a"
server_key_iv: "d1a4deeb5972e161db3caeb78f5eea3d"
server_key_version: "encryptionkey@0"
I20260812 16:52:15.393445 18155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 16:52:15.397415 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:15.399024 18155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:15.399227 18155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "1c553275664a44f6a1ebdef169e3f8d0"
format_stamp: "Formatted at 2026-08-12 16:52:15 on dist-test-slave-z8x9"
server_key: "db82b6ea5115acfbb3d07eebb3d6966a"
server_key_iv: "d1a4deeb5972e161db3caeb78f5eea3d"
server_key_version: "encryptionkey@0"
I20260812 16:52:15.399469 18155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:15.428308 18155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:15.432960 18155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:15.433329 18155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:15.434357 18155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:15.435847 18155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:15.435938 18155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:15.436100 18155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:15.436193 18155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:15.440107 17900 heartbeater.cc:503] Master 127.8.195.190:35337 was elected leader, sending a full tablet report...
I20260812 16:52:15.473441 18155 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:40749
I20260812 16:52:15.473529 18409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:40749 every 8 connection(s)
I20260812 16:52:15.474395 18155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:15.483193 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18155
I20260812 16:52:15.483434 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:15.483975 8974 external_mini_cluster.cc:1428] Setting key f1a89cc07b3f86d199fa54c199fcbc40
Aug 12 16:52:15 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553535, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:15.496150 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.477083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:40065 (local address 127.8.195.190:35337)
0812 16:52:15.477644 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.477649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.478439 (+ 790us) server_negotiation.cc:407] Connection header received
0812 16:52:15.480357 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.480362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.480473 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.480598 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:15.482344 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.483371 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.485664 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.486366 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.486486 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.490516 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:15.490521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:15.490524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:15.490557 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:15.493149 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.493936 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.493940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.493942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.494067 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.494516 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.494519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.494522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.494803 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:15.494921 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.495672 (+ 751us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.495875 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":320,"threads_started":1}
I20260812 16:52:15.497413 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.477529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35337 (local address 127.8.195.131:40065)
0812 16:52:15.478262 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.478301 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.480067 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.480849 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.480858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.481510 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:15.482109 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.482126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.483564 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.483570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.485489 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.485507 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.486615 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.486622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.486930 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.487831 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:15.487854 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:15.490308 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:15.493350 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:15.493357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:15.493370 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:15.493814 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:15.494175 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:15.494206 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:15.494209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:15.494403 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:15.494923 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:15.494935 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:15.495312 (+ 377us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.495563 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.495999 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":298,"threads_started":1}
I20260812 16:52:15.498999 18412 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35337
I20260812 16:52:15.499342 18412 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:15.500141 18412 heartbeater.cc:511] Master 127.8.195.190:35337 requested a full tablet report, sending...
I20260812 16:52:15.501709 17525 ts_manager.cc:194] Registered new tserver with Master: 1c553275664a44f6a1ebdef169e3f8d0 (127.8.195.131:40749)
I20260812 16:52:15.502616 17525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:40065
I20260812 16:52:15.512877 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:52:15.532210 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.518236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.8.195.190:35337)
0812 16:52:15.518449 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.518454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.518853 (+ 399us) server_negotiation.cc:407] Connection header received
0812 16:52:15.518993 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.518997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.519073 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.519166 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:15.520699 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.521558 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.522980 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.523451 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.523564 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.526241 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:15.526253 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:15.526256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:15.526300 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:15.529341 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.530128 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.530133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.530135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.530289 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.530739 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.530742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.530745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.531107 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:15.531324 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.531730 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.531961 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 16:52:15.537387 17525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:15.544724 17525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:15.577590 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.558440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40749 (local address 127.0.0.1:36342)
0812 16:52:15.559425 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.559489 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.559714 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.569853 (+ 10139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.569861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.569896 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:15.570490 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.570506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.573022 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.573028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.574934 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.574951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.576321 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.576329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.576580 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.576634 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.576841 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.577080 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":428,"threads_started":1}
I20260812 16:52:15.577694 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.558644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.8.195.131:40749)
0812 16:52:15.569007 (+ 10363us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.569016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.569050 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:52:15.569186 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.569198 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.569522 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.570727 (+ 1205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:15.571124 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.572853 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.575100 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.576095 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.576254 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.576538 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.576923 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.577256 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10152,"thread_start_us":10232,"threads_started":1}
I20260812 16:52:15.580009 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.560176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.8.195.130:36355)
0812 16:52:15.569611 (+ 9435us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.569620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.569652 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:52:15.569771 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.569778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.570026 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.571641 (+ 1615us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:15.571900 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.573449 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.576547 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.578928 (+ 2381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.579106 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.579323 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.579519 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.579675 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9233,"thread_start_us":9133,"threads_started":1}
I20260812 16:52:15.580057 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.559858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:36355 (local address 127.0.0.1:60446)
0812 16:52:15.568595 (+ 8737us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.568623 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.568793 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.570935 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.570952 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.570975 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:15.571424 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.571439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.573610 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.573617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.576297 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.576317 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.579139 (+ 2822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.579145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.579318 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.579358 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.579477 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.579610 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8556,"mutex_wait_us":1,"thread_start_us":10465,"threads_started":1}
I20260812 16:52:15.581255 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.560481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37131 (local address 127.0.0.1:46122)
0812 16:52:15.570642 (+ 10161us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.570664 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.570770 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.571647 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.571652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.571682 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:15.572147 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.572164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.574467 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.574474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.577090 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.577110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.580387 (+ 3277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.580397 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.580578 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.580611 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.580722 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.580863 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10010,"thread_start_us":7727,"threads_started":1}
I20260812 16:52:15.581318 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.561151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.8.195.129:37131)
0812 16:52:15.571140 (+ 9989us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.571147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.571173 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:52:15.571257 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.571262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.571430 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.571665 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:15.572422 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.574216 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.577465 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.579522 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.579686 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.579937 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.580791 (+ 854us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.580964 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9772,"thread_start_us":9568,"threads_started":1}
I20260812 16:52:15.582640 18283 tablet_service.cc:1467] Processing CreateTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (DEFAULT_TABLE table=test-table [id=7cc18d6835534fe59de68a3b315ad817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:15.583847 18025 tablet_service.cc:1467] Processing CreateTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (DEFAULT_TABLE table=test-table [id=7cc18d6835534fe59de68a3b315ad817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:15.584728 18283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:15.585227 18025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:15.585285 17770 tablet_service.cc:1467] Processing CreateTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (DEFAULT_TABLE table=test-table [id=7cc18d6835534fe59de68a3b315ad817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:15.586603 17770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:15.597762 18450 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Bootstrap starting.
I20260812 16:52:15.600237 18451 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Bootstrap starting.
I20260812 16:52:15.602461 18453 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Bootstrap starting.
I20260812 16:52:15.603497 18450 tablet_bootstrap.cc:654] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:15.604573 18451 tablet_bootstrap.cc:654] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:15.604789 18450 log.cc:824] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:15.606019 18451 log.cc:824] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:15.606904 18453 tablet_bootstrap.cc:654] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:15.608004 18450 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: No bootstrap required, opened a new log
I20260812 16:52:15.608292 18450 ts_tablet_manager.cc:1397] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260812 16:52:15.608237 18453 log.cc:824] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:15.609442 18451 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: No bootstrap required, opened a new log
I20260812 16:52:15.609784 18451 ts_tablet_manager.cc:1397] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 16:52:15.612504 18453 tablet_bootstrap.cc:492] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: No bootstrap required, opened a new log
I20260812 16:52:15.612802 18453 ts_tablet_manager.cc:1397] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260812 16:52:15.616683 18450 raft_consensus.cc:348] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.617053 18450 raft_consensus.cc:374] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:15.617203 18450 raft_consensus.cc:729] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c553275664a44f6a1ebdef169e3f8d0, State: Initialized, Role: FOLLOWER
I20260812 16:52:15.617830 18450 consensus_queue.cc:260] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.619009 18412 heartbeater.cc:503] Master 127.8.195.190:35337 was elected leader, sending a full tablet report...
I20260812 16:52:15.619685 18450 ts_tablet_manager.cc:1428] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Time spent starting tablet: real 0.011s user 0.012s sys 0.001s
I20260812 16:52:15.621767 18451 raft_consensus.cc:348] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.622256 18451 raft_consensus.cc:374] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:15.622377 18451 raft_consensus.cc:729] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d065be6f765549a998c1254b2af281b8, State: Initialized, Role: FOLLOWER
I20260812 16:52:15.623059 18451 consensus_queue.cc:260] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.624584 18145 heartbeater.cc:503] Master 127.8.195.190:35337 was elected leader, sending a full tablet report...
I20260812 16:52:15.625192 18451 ts_tablet_manager.cc:1428] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260812 16:52:15.624825 18453 raft_consensus.cc:348] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.625379 18453 raft_consensus.cc:374] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:15.625500 18453 raft_consensus.cc:729] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6022b68377ff41a5bd397abe169838cc, State: Initialized, Role: FOLLOWER
I20260812 16:52:15.626153 18453 consensus_queue.cc:260] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.628409 18453 ts_tablet_manager.cc:1428] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260812 16:52:15.655864 17901 tablet.cc:2369] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:52:15.682309 18146 tablet.cc:2369] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:52:15.726078 18413 tablet.cc:2369] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:52:15.733783 18462 raft_consensus.cc:482] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:15.734138 18462 raft_consensus.cc:504] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.736268 18462 leader_election.cc:288] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c553275664a44f6a1ebdef169e3f8d0 (127.8.195.131:40749), 6022b68377ff41a5bd397abe169838cc (127.8.195.129:37131)
I20260812 16:52:15.744809 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.736791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:52443 (local address 127.8.195.129:37131)
0812 16:52:15.737004 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.737011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.737895 (+ 884us) server_negotiation.cc:407] Connection header received
0812 16:52:15.738252 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.738261 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.738357 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.738458 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:15.739045 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.740025 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.742491 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.743773 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.743931 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.744002 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.744395 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.744537 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:52:15.744889 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.736667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37131 (local address 127.8.195.130:52443)
0812 16:52:15.737779 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.737803 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.738028 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.738471 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.738477 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.738505 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:15.738872 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.738882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.740165 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.740209 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.742307 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.742323 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.744025 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.744032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.744200 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.744236 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.744334 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.744484 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":23,"thread_start_us":742,"threads_started":1}
I20260812 16:52:15.745609 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.736794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:59825 (local address 127.8.195.131:40749)
0812 16:52:15.736994 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.736999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.738351 (+ 1352us) server_negotiation.cc:407] Connection header received
0812 16:52:15.738579 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.738585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.738748 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.738827 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:15.739683 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.740810 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.743155 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.744201 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.744377 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.744447 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.744888 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.745355 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:52:15.745738 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.737373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40749 (local address 127.8.195.130:59825)
0812 16:52:15.738220 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:15.738239 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:15.738396 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:15.739058 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:15.739063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:15.739094 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:15.739511 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.739523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.740970 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.740977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.742891 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:15.742903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.744449 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.744456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:15.744668 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.744702 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:52:15.744836 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 16:52:15.745382 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":4,"thread_start_us":474,"threads_started":1}
I20260812 16:52:15.746140 17810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0247d0d3fb044f7cb3dbe5c4ef4cfebd" candidate_uuid: "d065be6f765549a998c1254b2af281b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022b68377ff41a5bd397abe169838cc" is_pre_election: true
I20260812 16:52:15.746779 17810 raft_consensus.cc:2454] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d065be6f765549a998c1254b2af281b8 in term 0.
I20260812 16:52:15.746914 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0247d0d3fb044f7cb3dbe5c4ef4cfebd" candidate_uuid: "d065be6f765549a998c1254b2af281b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c553275664a44f6a1ebdef169e3f8d0" is_pre_election: true
I20260812 16:52:15.747406 18325 raft_consensus.cc:2454] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d065be6f765549a998c1254b2af281b8 in term 0.
I20260812 16:52:15.747608 17941 leader_election.cc:302] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022b68377ff41a5bd397abe169838cc, d065be6f765549a998c1254b2af281b8; no voters:
I20260812 16:52:15.748070 18462 raft_consensus.cc:2781] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:15.748238 18462 raft_consensus.cc:482] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:15.748343 18462 raft_consensus.cc:3037] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:15.750653 18462 raft_consensus.cc:504] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.751353 18462 leader_election.cc:288] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [CANDIDATE]: Term 1 election: Requested vote from peers 1c553275664a44f6a1ebdef169e3f8d0 (127.8.195.131:40749), 6022b68377ff41a5bd397abe169838cc (127.8.195.129:37131)
I20260812 16:52:15.751842 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0247d0d3fb044f7cb3dbe5c4ef4cfebd" candidate_uuid: "d065be6f765549a998c1254b2af281b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c553275664a44f6a1ebdef169e3f8d0"
I20260812 16:52:15.751935 17810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0247d0d3fb044f7cb3dbe5c4ef4cfebd" candidate_uuid: "d065be6f765549a998c1254b2af281b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022b68377ff41a5bd397abe169838cc"
I20260812 16:52:15.752031 18325 raft_consensus.cc:3037] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:15.752233 17810 raft_consensus.cc:3037] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:15.754642 18325 raft_consensus.cc:2454] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d065be6f765549a998c1254b2af281b8 in term 1.
I20260812 16:52:15.754946 17810 raft_consensus.cc:2454] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d065be6f765549a998c1254b2af281b8 in term 1.
I20260812 16:52:15.755115 17937 leader_election.cc:302] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c553275664a44f6a1ebdef169e3f8d0, d065be6f765549a998c1254b2af281b8; no voters:
I20260812 16:52:15.755440 18462 raft_consensus.cc:2781] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:15.755996 18462 raft_consensus.cc:686] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 1 LEADER]: Becoming Leader. State: Replica: d065be6f765549a998c1254b2af281b8, State: Running, Role: LEADER
I20260812 16:52:15.756565 18462 consensus_queue.cc:237] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } }
I20260812 16:52:15.761744 17525 catalog_manager.cc:5495] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d065be6f765549a998c1254b2af281b8 (127.8.195.130). New cstate: current_term: 1 leader_uuid: "d065be6f765549a998c1254b2af281b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d065be6f765549a998c1254b2af281b8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 36355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 } health_report { overall_health: UNKNOWN } } }
I20260812 16:52:15.814502 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.808597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.8.195.130:36355)
0812 16:52:15.808791 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.808795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.808818 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:15.809018 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.809022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.809109 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.809282 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:15.809701 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.810890 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.812247 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.812666 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.812802 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.813361 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:15.813888 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:15.814072 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.814121 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.814209 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:52:15.822948 18325 raft_consensus.cc:1261] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 1 FOLLOWER]: Refusing update from remote peer d065be6f765549a998c1254b2af281b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:15.822949 17810 raft_consensus.cc:1261] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 1 FOLLOWER]: Refusing update from remote peer d065be6f765549a998c1254b2af281b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:15.823944 18462 consensus_queue.cc:1035] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6022b68377ff41a5bd397abe169838cc" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:15.824249 18470 consensus_queue.cc:1035] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c553275664a44f6a1ebdef169e3f8d0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:15.835987 18477 mvcc.cc:205] Tried to move back new op lower bound from 7317723282719588352 to 7317723282463354880. Current Snapshot: MvccSnapshot[applied={T|T < 7317723282719588352}]
I20260812 16:52:15.839725 18479 mvcc.cc:205] Tried to move back new op lower bound from 7317723282719588352 to 7317723282463354880. Current Snapshot: MvccSnapshot[applied={T|T < 7317723282719588352}]
I20260812 16:52:15.850745 17525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:15.851114 17525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:15.855334 17525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 802f99f41d0542f985fae5583fdec55e: Sending DeleteTablet for 3 replicas of tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd
I20260812 16:52:15.856878 18283 tablet_service.cc:1514] Processing DeleteTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:15 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:36342
I20260812 16:52:15.857013 18025 tablet_service.cc:1514] Processing DeleteTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:15 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:60446
I20260812 16:52:15.857822 17770 tablet_service.cc:1514] Processing DeleteTablet for tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:15 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:46122
I20260812 16:52:15.858945 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17659
I20260812 16:52:15.867871 18497 tablet_replica.cc:330] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: stopping tablet replica
I20260812 16:52:15.868604 18497 raft_consensus.cc:2229] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:15.869714 18489 tablet_replica.cc:330] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: stopping tablet replica
I20260812 16:52:15.870103 18497 raft_consensus.cc:2258] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:15.870165 18489 raft_consensus.cc:2229] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:15.870162 18501 tablet_replica.cc:330] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: stopping tablet replica
I20260812 16:52:15.870823 18501 raft_consensus.cc:2229] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:15.870940 18489 raft_consensus.cc:2258] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:15.871503 18501 raft_consensus.cc:2258] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:15.872442 18497 ts_tablet_manager.cc:1905] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:15.873332 18489 ts_tablet_manager.cc:1905] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:15.876065 18501 ts_tablet_manager.cc:1905] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:15.880425 18489 ts_tablet_manager.cc:1918] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:15.880612 18489 log.cc:1196] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/0247d0d3fb044f7cb3dbe5c4ef4cfebd
I20260812 16:52:15.880899 18497 ts_tablet_manager.cc:1918] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:15.881100 18497 log.cc:1196] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/0247d0d3fb044f7cb3dbe5c4ef4cfebd
I20260812 16:52:15.881213 18489 ts_tablet_manager.cc:1939] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 1c553275664a44f6a1ebdef169e3f8d0: Deleting consensus metadata
I20260812 16:52:15.881789 18497 ts_tablet_manager.cc:1939] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P d065be6f765549a998c1254b2af281b8: Deleting consensus metadata
I20260812 16:52:15.883179 17492 catalog_manager.cc:4841] TS 1c553275664a44f6a1ebdef169e3f8d0 (127.8.195.131:40749): tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (table test-table [id=7cc18d6835534fe59de68a3b315ad817]) successfully deleted
I20260812 16:52:15.883651 17492 catalog_manager.cc:4841] TS d065be6f765549a998c1254b2af281b8 (127.8.195.130:36355): tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (table test-table [id=7cc18d6835534fe59de68a3b315ad817]) successfully deleted
I20260812 16:52:15.886477 18501 ts_tablet_manager.cc:1918] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:15.886683 18501 log.cc:1196] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/0247d0d3fb044f7cb3dbe5c4ef4cfebd
I20260812 16:52:15.887270 18501 ts_tablet_manager.cc:1939] T 0247d0d3fb044f7cb3dbe5c4ef4cfebd P 6022b68377ff41a5bd397abe169838cc: Deleting consensus metadata
I20260812 16:52:15.889122 17495 catalog_manager.cc:4841] TS 6022b68377ff41a5bd397abe169838cc (127.8.195.129:37131): tablet 0247d0d3fb044f7cb3dbe5c4ef4cfebd (table test-table [id=7cc18d6835534fe59de68a3b315ad817]) successfully deleted
Aug 12 16:52:15 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:15.898382 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:15.860886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.8.195.129:37131)
0812 16:52:15.861123 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:15.861133 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:15.872768 (+ 11635us) server_negotiation.cc:407] Connection header received
0812 16:52:15.873017 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:15.873023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:15.873123 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:15.873265 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:15.881562 (+ 8297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.882639 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.884430 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:15.885316 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:15.885465 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:15.890506 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:15.890521 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:15.890543 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:15.890588 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:15.894808 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.895571 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.895581 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.895585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.895781 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:15.896166 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:15.896172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:15.896176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:15.896732 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:15.897049 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:15.897669 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 16:52:15.897957 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 16:52:15.899190 17893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:16.011899 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17659
I20260812 16:52:16.051411 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17909
Aug 12 16:52:16 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:52:16.069854 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:16.052247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.8.195.130:36355)
0812 16:52:16.052432 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:16.052436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:16.052454 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:52:16.052662 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:16.052666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:16.052738 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:16.052839 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:16.054514 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:16.055378 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:16.057281 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:16.057766 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:16.057875 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:16.063130 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:16.063148 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:16.063165 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:16.063210 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:16.066593 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:16.067372 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:16.067381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:16.067383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:16.067548 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:16.067995 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:16.067998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:16.068001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:16.068462 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:16.068826 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:16.069213 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 16:52:16.069469 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:52:16.070528 18136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:16.187826 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17909
I20260812 16:52:16.215183 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18155
Aug 12 16:52:16 dist-test-slave-z8x9 krb5kdc[17437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:52:16.232134 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:16.216016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.8.195.131:40749)
0812 16:52:16.216230 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:16.216237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:16.216341 (+ 104us) server_negotiation.cc:407] Connection header received
0812 16:52:16.216568 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:16.216578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:16.216661 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:16.216772 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:16.218225 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:16.218953 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:16.220251 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:16.220696 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:16.220842 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:16.225961 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:16.225979 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:16.225996 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:16.226037 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:16.229356 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:16.230048 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:16.230060 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:16.230062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:16.230229 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:16.230719 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:16.230723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:16.230726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:16.231126 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:16.231385 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:16.231669 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 16:52:16.231862 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 16:52:16.232904 18402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:16.324379 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18155
I20260812 16:52:16.352736 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17458
I20260812 16:52:16.354015 17591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:16.465857 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17458
2026-08-12T16:52:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5362 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T16:52:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:16Z Disabled control of system clock
I20260812 16:52:16.557678 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42115
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:52:16.897477 18521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:16.897804 18521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:16.897984 18521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:16.905400 18521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:16.905519 18521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:16.905599 18521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:16.905681 18521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:16.905741 18521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:16.915218 18521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:16.916740 18521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:16.918666 18521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:16.930850 18534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:16.930817 18539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:16.931088 18529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:16.931845 18521 server_base.cc:1034] running on GCE node
I20260812 16:52:16.932703 18521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:16.934266 18521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:16.935554 18521 hybrid_clock.cc:648] HybridClock initialized: now 1786553536935494 us; error 49 us; skew 500 ppm
I20260812 16:52:16.935936 18521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:16.938313 18521 webserver.cc:460] Webserver started at http://127.8.195.190:32935/ using document root <none> and password file <none>
I20260812 16:52:16.939054 18521 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:16.939195 18521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:16.939543 18521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:16.942627 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "46cf13872d4248b3bf815faaea06271f"
format_stamp: "Formatted at 2026-08-12 16:52:16 on dist-test-slave-z8x9"
server_key: "b5d00faeee2d00799a3e7dd0d641c4cb"
server_key_iv: "a82025085de23b915ec19d9104a7dc8b"
server_key_version: "encryptionkey@0"
I20260812 16:52:16.943475 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "46cf13872d4248b3bf815faaea06271f"
format_stamp: "Formatted at 2026-08-12 16:52:16 on dist-test-slave-z8x9"
server_key: "b5d00faeee2d00799a3e7dd0d641c4cb"
server_key_iv: "a82025085de23b915ec19d9104a7dc8b"
server_key_version: "encryptionkey@0"
I20260812 16:52:16.948848 18521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:52:16.952471 18547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:16.953912 18521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:52:16.954115 18521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "46cf13872d4248b3bf815faaea06271f"
format_stamp: "Formatted at 2026-08-12 16:52:16 on dist-test-slave-z8x9"
server_key: "b5d00faeee2d00799a3e7dd0d641c4cb"
server_key_iv: "a82025085de23b915ec19d9104a7dc8b"
server_key_version: "encryptionkey@0"
I20260812 16:52:16.954367 18521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:16.960450 18521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:16.961426 18521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:16.961750 18521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:16.980904 18521 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:42115
I20260812 16:52:16.980942 18654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:42115 every 8 connection(s)
I20260812 16:52:16.982409 18521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:16.986476 18657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:16.990672 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18521
I20260812 16:52:16.990890 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:16.991766 8974 external_mini_cluster.cc:1428] Setting key 9ffa2584c4072a53b01457fafc6beee1
I20260812 16:52:16.999270 18657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: Bootstrap starting.
I20260812 16:52:17.001427 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:16.994139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35030 (local address 127.8.195.190:42115)
0812 16:52:16.995211 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:16.995223 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:16.995280 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:52:16.997404 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:16.997431 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:16.997884 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:16.998239 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:16.999660 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:16.999666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:16.999679 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:16.999695 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:16.999766 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:16.999786 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:16.999964 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:17.000103 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 16:52:17.000376 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":422,"threads_started":1}
I20260812 16:52:17.002851 18657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:17.003928 18657 log.cc:824] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:17.006700 18657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: No bootstrap required, opened a new log
I20260812 16:52:17.013958 18657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } }
I20260812 16:52:17.014372 18657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:17.014487 18657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46cf13872d4248b3bf815faaea06271f, State: Initialized, Role: FOLLOWER
I20260812 16:52:17.015053 18657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } }
I20260812 16:52:17.015312 18657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:17.015460 18657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:17.015635 18657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:17.017984 18657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } }
I20260812 16:52:17.018540 18657 leader_election.cc:302] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46cf13872d4248b3bf815faaea06271f; no voters:
I20260812 16:52:17.019294 18657 leader_election.cc:288] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:17.019459 18664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:17.020915 18664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [term 1 LEADER]: Becoming Leader. State: Replica: 46cf13872d4248b3bf815faaea06271f, State: Running, Role: LEADER
I20260812 16:52:17.021806 18657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:17.021926 18664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } }
I20260812 16:52:17.024770 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46cf13872d4248b3bf815faaea06271f. Latest consensus state: current_term: 1 leader_uuid: "46cf13872d4248b3bf815faaea06271f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } } }
I20260812 16:52:17.025197 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:17.025740 18666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46cf13872d4248b3bf815faaea06271f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46cf13872d4248b3bf815faaea06271f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42115 } } }
I20260812 16:52:17.026134 18666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:17.027930 18684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:17.033093 18684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:17.042457 18684 catalog_manager.cc:1347] Generated new cluster ID: bee90e3d1fd1488eaed409a6f25eaadb
I20260812 16:52:17.042587 18684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:17.063292 18684 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:17.063800 18684 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:17.081933 18684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: Generated new TSK 0
I20260812 16:52:17.083060 18684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:52:17.090823 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--builtin_ntp_servers=127.8.195.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 16:52:17.435621 18705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:17.436049 18705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:17.436220 18705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:17.443579 18705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:17.443746 18705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:17.443933 18705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:17.453670 18705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:17.455052 18705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:17.457000 18705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:17.471174 18712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:17.471275 18722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:17.471194 18711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:17.472105 18705 server_base.cc:1034] running on GCE node
I20260812 16:52:17.472756 18705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:17.473922 18705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:17.475162 18705 hybrid_clock.cc:648] HybridClock initialized: now 1786553537475118 us; error 59 us; skew 500 ppm
I20260812 16:52:17.475517 18705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:17.477952 18705 webserver.cc:460] Webserver started at http://127.8.195.129:44701/ using document root <none> and password file <none>
I20260812 16:52:17.478689 18705 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:17.478834 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:17.479174 18705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:17.482255 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "9e5b33da53584738840adfde581a87b7"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "48a0d6ae275d857dcc49ee3ce83ab412"
server_key_iv: "4fd9b1de2fca322bc8409c11bfb91a16"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.483095 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "9e5b33da53584738840adfde581a87b7"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "48a0d6ae275d857dcc49ee3ce83ab412"
server_key_iv: "4fd9b1de2fca322bc8409c11bfb91a16"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.488426 18705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 16:52:17.491904 18733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:17.493233 18705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:17.493429 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "9e5b33da53584738840adfde581a87b7"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "48a0d6ae275d857dcc49ee3ce83ab412"
server_key_iv: "4fd9b1de2fca322bc8409c11bfb91a16"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.493691 18705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:17.499555 18705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:17.500440 18705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:17.500793 18705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:17.501843 18705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:17.503293 18705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:17.503389 18705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:17.503545 18705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:17.503629 18705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:17.538720 18705 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:46387
I20260812 16:52:17.538779 18954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:46387 every 8 connection(s)
I20260812 16:52:17.540155 18705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:17.548413 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18705
I20260812 16:52:17.548660 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:17.549211 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:17.543044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:35401 (local address 127.8.195.190:42115)
0812 16:52:17.543524 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:17.543529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:17.544840 (+ 1311us) server_negotiation.cc:407] Connection header received
0812 16:52:17.546666 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:17.546671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:17.546738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:17.546847 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:17.548337 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:17.548351 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:17.548355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:17.548373 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:17.548420 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:17.548432 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:17.548583 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:17.548797 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 16:52:17.548873 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":259,"threads_started":1}
I20260812 16:52:17.549494 8974 external_mini_cluster.cc:1428] Setting key 628afc840d77af57e663c416c2109e38
I20260812 16:52:17.550300 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:17.543689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42115 (local address 127.8.195.129:35401)
0812 16:52:17.544655 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:17.544703 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:17.546356 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:17.547067 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:17.547076 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:17.547171 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:17.547954 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:17.547957 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:17.548003 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:17.548099 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:17.548542 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:17.548555 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:17.548565 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:52:17.548890 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 16:52:17.549206 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":282,"threads_started":1}
I20260812 16:52:17.552009 18957 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42115
I20260812 16:52:17.552320 18957 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:17.552714 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--builtin_ntp_servers=127.8.195.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:52:17.552987 18957 heartbeater.cc:511] Master 127.8.195.190:42115 requested a full tablet report, sending...
I20260812 16:52:17.555385 18579 ts_manager.cc:194] Registered new tserver with Master: 9e5b33da53584738840adfde581a87b7 (127.8.195.129:46387)
W20260812 16:52:17.892216 18963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:17.892571 18963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:17.892725 18963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:17.902473 18963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:17.902660 18963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:17.902889 18963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:17.915148 18963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:17.916425 18963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:17.918267 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:17.932319 18977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:17.932564 18975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:17.932564 18974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:17.933760 18963 server_base.cc:1034] running on GCE node
I20260812 16:52:17.934465 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:17.935477 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:17.936686 18963 hybrid_clock.cc:648] HybridClock initialized: now 1786553537936643 us; error 40 us; skew 500 ppm
I20260812 16:52:17.937027 18963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:17.939386 18963 webserver.cc:460] Webserver started at http://127.8.195.130:39975/ using document root <none> and password file <none>
I20260812 16:52:17.940097 18963 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:17.940215 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:17.940558 18963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:17.944614 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "2f92ccb77dae4ee19a1554110a01f9c9"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "5b0696def5043696060b30e3683602f7"
server_key_iv: "ac3972024c410c7bc5d316c4b257a8b8"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.945559 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "2f92ccb77dae4ee19a1554110a01f9c9"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "5b0696def5043696060b30e3683602f7"
server_key_iv: "ac3972024c410c7bc5d316c4b257a8b8"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.950928 18963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:17.955247 18989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:17.956661 18963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:52:17.956858 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "2f92ccb77dae4ee19a1554110a01f9c9"
format_stamp: "Formatted at 2026-08-12 16:52:17 on dist-test-slave-z8x9"
server_key: "5b0696def5043696060b30e3683602f7"
server_key_iv: "ac3972024c410c7bc5d316c4b257a8b8"
server_key_version: "encryptionkey@0"
I20260812 16:52:17.957114 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:17.963428 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:17.964308 18963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:17.964612 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:17.965684 18963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:17.967218 18963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:17.967382 18963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:17.967525 18963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:17.967620 18963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:18.006584 18963 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:41547
I20260812 16:52:18.006656 19193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:41547 every 8 connection(s)
I20260812 16:52:18.008042 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:18.009398 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18963
I20260812 16:52:18.009585 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:18.010217 8974 external_mini_cluster.cc:1428] Setting key 712cbcf4df2e1cbc2c211ac9421c28dd
I20260812 16:52:18.013711 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--builtin_ntp_servers=127.8.195.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 16:52:18.017227 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.011006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:40355 (local address 127.8.195.190:42115)
0812 16:52:18.011234 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.011239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.012967 (+ 1728us) server_negotiation.cc:407] Connection header received
0812 16:52:18.014758 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.014767 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.014864 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.014987 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.016328 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.016337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.016344 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.016377 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.016424 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.016442 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.016584 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.016752 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.016823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:52:18.018116 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.011656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42115 (local address 127.8.195.130:40355)
0812 16:52:18.012803 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.012848 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.014487 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.015173 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.015180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.015270 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.016009 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.016011 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.016049 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.016135 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.016551 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.016558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.016565 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.016791 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.016989 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":321,"threads_started":1}
I20260812 16:52:18.019536 19194 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42115
I20260812 16:52:18.019855 19194 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:18.020486 19194 heartbeater.cc:511] Master 127.8.195.190:42115 requested a full tablet report, sending...
I20260812 16:52:18.022112 18579 ts_manager.cc:194] Registered new tserver with Master: 2f92ccb77dae4ee19a1554110a01f9c9 (127.8.195.130:41547)
W20260812 16:52:18.355111 19200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:18.355454 19200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:18.355612 19200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:18.363240 19200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:18.363387 19200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:18.363570 19200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:18.372608 19200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:18.373980 19200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:18.375821 19200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:18.389093 19216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:18.389271 19221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:18.389212 19219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:18.390362 19200 server_base.cc:1034] running on GCE node
I20260812 16:52:18.391005 19200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:18.392014 19200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:18.393272 19200 hybrid_clock.cc:648] HybridClock initialized: now 1786553538393219 us; error 57 us; skew 500 ppm
I20260812 16:52:18.393618 19200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:18.395929 19200 webserver.cc:460] Webserver started at http://127.8.195.131:35083/ using document root <none> and password file <none>
I20260812 16:52:18.396639 19200 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:18.396781 19200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:18.397137 19200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:18.400346 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "69488bb83123452799175a47d48b29e2"
format_stamp: "Formatted at 2026-08-12 16:52:18 on dist-test-slave-z8x9"
server_key: "73b6e8c0eaa738f0b537ffe94018ce3f"
server_key_iv: "e02736274311b6ba12ee56dc2a814815"
server_key_version: "encryptionkey@0"
I20260812 16:52:18.401121 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "69488bb83123452799175a47d48b29e2"
format_stamp: "Formatted at 2026-08-12 16:52:18 on dist-test-slave-z8x9"
server_key: "73b6e8c0eaa738f0b537ffe94018ce3f"
server_key_iv: "e02736274311b6ba12ee56dc2a814815"
server_key_version: "encryptionkey@0"
I20260812 16:52:18.406225 19200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:52:18.409618 19232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:18.410918 19200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:52:18.411109 19200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "69488bb83123452799175a47d48b29e2"
format_stamp: "Formatted at 2026-08-12 16:52:18 on dist-test-slave-z8x9"
server_key: "73b6e8c0eaa738f0b537ffe94018ce3f"
server_key_iv: "e02736274311b6ba12ee56dc2a814815"
server_key_version: "encryptionkey@0"
I20260812 16:52:18.411353 19200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:18.417073 19200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:18.417901 19200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:18.418177 19200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:18.419094 19200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:18.420614 19200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:18.420740 19200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:18.420879 19200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:18.420970 19200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:18.456182 19200 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:43717
I20260812 16:52:18.456280 19434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:43717 every 8 connection(s)
I20260812 16:52:18.457726 19200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:18.460834 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19200
I20260812 16:52:18.461004 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:18.461630 8974 external_mini_cluster.cc:1428] Setting key 599cc2eac08d12da9f1dd5c36a32e415
I20260812 16:52:18.467377 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.460555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:33179 (local address 127.8.195.190:42115)
0812 16:52:18.460742 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.460746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.462768 (+ 2022us) server_negotiation.cc:407] Connection header received
0812 16:52:18.464866 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.464873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.464949 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.465072 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.466710 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.466716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.466720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.466739 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.466778 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.466793 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.466911 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.467091 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.467141 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:52:18.468324 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.461338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42115 (local address 127.8.195.131:33179)
0812 16:52:18.462617 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.462655 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.464436 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.465518 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.465527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.465619 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.466287 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.466290 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.466342 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.466499 (+ 157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.466899 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.466908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.466913 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.467136 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.467342 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":374,"threads_started":1}
I20260812 16:52:18.469655 19436 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42115
I20260812 16:52:18.469990 19436 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:18.470598 19436 heartbeater.cc:511] Master 127.8.195.190:42115 requested a full tablet report, sending...
I20260812 16:52:18.471971 18579 ts_manager.cc:194] Registered new tserver with Master: 69488bb83123452799175a47d48b29e2 (127.8.195.131:43717)
I20260812 16:52:18.477433 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:52:18.484751 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.482219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.8.195.190:42115)
0812 16:52:18.482423 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.482428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.482964 (+ 536us) server_negotiation.cc:407] Connection header received
0812 16:52:18.483112 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.483116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.483176 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.483266 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.484191 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.484195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.484198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.484213 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.484246 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.484257 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.484359 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.484492 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.484542 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:52:18.488765 18579 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:18.496225 18579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:18.525022 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.508825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41547 (local address 127.0.0.1:37590)
0812 16:52:18.520429 (+ 11604us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.520536 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.520725 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.522446 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.522454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.522549 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.523611 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.523616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.523653 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.523774 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.524264 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.524276 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.524284 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.524484 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.524585 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11315,"thread_start_us":12262,"threads_started":1}
I20260812 16:52:18.525805 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.509385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.8.195.131:43717)
0812 16:52:18.519999 (+ 10614us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.520011 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.521738 (+ 1727us) server_negotiation.cc:407] Connection header received
0812 16:52:18.521848 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.521856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.522132 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.522421 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.524695 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.524705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.524720 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.524741 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.524798 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.524819 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.524965 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.525293 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.525400 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10401,"thread_start_us":10208,"threads_started":1}
I20260812 16:52:18.526316 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.509235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.8.195.130:41547)
0812 16:52:18.521827 (+ 12592us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.521838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.521872 (+ 34us) server_negotiation.cc:407] Connection header received
0812 16:52:18.521950 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.521958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.522203 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.522430 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.523999 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.524011 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.524029 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.524053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.524123 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.524153 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.524307 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.525882 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.525990 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12403,"thread_start_us":12268,"threads_started":1}
I20260812 16:52:18.526845 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.508947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:43717 (local address 127.0.0.1:40388)
0812 16:52:18.521396 (+ 12449us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.521416 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.521528 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.523472 (+ 1944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.523478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.523511 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.524442 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.524447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.524467 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.524513 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.525085 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.525093 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.525100 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.526327 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.526383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12319,"thread_start_us":11090,"threads_started":1}
I20260812 16:52:18.527426 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.521339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.8.195.129:46387)
0812 16:52:18.522071 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.522080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.524403 (+ 2323us) server_negotiation.cc:407] Connection header received
0812 16:52:18.524593 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.524601 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.524841 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.525074 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.526167 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.526174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.526185 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.526208 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.526254 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.526274 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.526405 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.526967 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.527089 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":463,"threads_started":1}
I20260812 16:52:18.528017 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.522110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:46387 (local address 127.0.0.1:54316)
0812 16:52:18.524197 (+ 2087us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.524215 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.524317 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.525203 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.525207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.525229 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.525924 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.525927 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.525945 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.525994 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.526457 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.526463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.526470 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.526609 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.526657 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1962,"thread_start_us":1823,"threads_started":1}
I20260812 16:52:18.530503 19322 tablet_service.cc:1467] Processing CreateTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 (DEFAULT_TABLE table=test-table [id=67ed73e5c361480394b2281af0584728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:18.530520 19077 tablet_service.cc:1467] Processing CreateTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 (DEFAULT_TABLE table=test-table [id=67ed73e5c361480394b2281af0584728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:18.531945 19322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e7c3a6c9854d7fb895a3341c2e6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:18.531967 19077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e7c3a6c9854d7fb895a3341c2e6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:18.532029 18823 tablet_service.cc:1467] Processing CreateTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 (DEFAULT_TABLE table=test-table [id=67ed73e5c361480394b2281af0584728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:18.533396 18823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e7c3a6c9854d7fb895a3341c2e6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:18.552037 19474 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Bootstrap starting.
I20260812 16:52:18.552237 19477 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Bootstrap starting.
I20260812 16:52:18.553372 19475 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Bootstrap starting.
I20260812 16:52:18.557016 19474 tablet_bootstrap.cc:654] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:18.557315 19475 tablet_bootstrap.cc:654] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:18.557870 19477 tablet_bootstrap.cc:654] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:18.558399 19474 log.cc:824] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:18.558739 18957 heartbeater.cc:503] Master 127.8.195.190:42115 was elected leader, sending a full tablet report...
I20260812 16:52:18.559221 19477 log.cc:824] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:18.560164 19475 log.cc:824] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:18.562480 19474 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: No bootstrap required, opened a new log
I20260812 16:52:18.562767 19474 ts_tablet_manager.cc:1397] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 16:52:18.566465 19475 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: No bootstrap required, opened a new log
I20260812 16:52:18.566833 19475 ts_tablet_manager.cc:1397] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.003s
I20260812 16:52:18.567152 19477 tablet_bootstrap.cc:492] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: No bootstrap required, opened a new log
I20260812 16:52:18.567422 19477 ts_tablet_manager.cc:1397] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260812 16:52:18.573037 19474 raft_consensus.cc:348] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.573547 19474 raft_consensus.cc:374] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:18.573661 19474 raft_consensus.cc:729] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69488bb83123452799175a47d48b29e2, State: Initialized, Role: FOLLOWER
I20260812 16:52:18.574321 19474 consensus_queue.cc:260] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.574530 19477 raft_consensus.cc:348] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.574997 19477 raft_consensus.cc:374] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:18.575133 19477 raft_consensus.cc:729] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5b33da53584738840adfde581a87b7, State: Initialized, Role: FOLLOWER
I20260812 16:52:18.575811 19477 consensus_queue.cc:260] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.577235 19436 heartbeater.cc:503] Master 127.8.195.190:42115 was elected leader, sending a full tablet report...
I20260812 16:52:18.577824 19474 ts_tablet_manager.cc:1428] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260812 16:52:18.577952 19475 raft_consensus.cc:348] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.578488 19475 raft_consensus.cc:374] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:18.578631 19475 raft_consensus.cc:729] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f92ccb77dae4ee19a1554110a01f9c9, State: Initialized, Role: FOLLOWER
I20260812 16:52:18.579342 19475 consensus_queue.cc:260] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.579735 19485 raft_consensus.cc:482] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:18.580084 19485 raft_consensus.cc:504] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.582211 19475 ts_tablet_manager.cc:1428] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260812 16:52:18.584581 19194 heartbeater.cc:503] Master 127.8.195.190:42115 was elected leader, sending a full tablet report...
I20260812 16:52:18.585573 19477 ts_tablet_manager.cc:1428] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260812 16:52:18.588073 19485 leader_election.cc:288] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f92ccb77dae4ee19a1554110a01f9c9 (127.8.195.130:41547), 69488bb83123452799175a47d48b29e2 (127.8.195.131:43717)
I20260812 16:52:18.589411 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.585040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:59361 (local address 127.8.195.130:41547)
0812 16:52:18.585265 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.585271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.586627 (+ 1356us) server_negotiation.cc:407] Connection header received
0812 16:52:18.586891 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.586896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.586953 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.587034 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.588624 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.588629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.588634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.588653 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.588691 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.588709 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.588795 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.589042 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.589105 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 16:52:18.590191 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.584503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41547 (local address 127.8.195.129:59361)
0812 16:52:18.586524 (+ 2021us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.586545 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.586736 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.587268 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.587276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.587304 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.588399 (+ 1095us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.588402 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.588420 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.588467 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.588892 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.588896 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.588902 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.589646 (+ 744us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.589709 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1883,"thread_start_us":3247,"threads_started":1}
I20260812 16:52:18.591233 19112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e7c3a6c9854d7fb895a3341c2e6217" candidate_uuid: "9e5b33da53584738840adfde581a87b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" is_pre_election: true
I20260812 16:52:18.592180 19112 raft_consensus.cc:2454] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b33da53584738840adfde581a87b7 in term 0.
I20260812 16:52:18.592881 18741 leader_election.cc:302] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f92ccb77dae4ee19a1554110a01f9c9, 9e5b33da53584738840adfde581a87b7; no voters:
I20260812 16:52:18.593408 19487 raft_consensus.cc:2781] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:18.593597 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.585163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:57239 (local address 127.8.195.131:43717)
0812 16:52:18.585379 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.585385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.590704 (+ 5319us) server_negotiation.cc:407] Connection header received
0812 16:52:18.590820 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.590828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.590910 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.591000 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.592228 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.592234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.592239 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.592259 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.592300 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.592315 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.592442 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.593254 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.593325 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:52:18.593612 19487 raft_consensus.cc:482] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:18.593647 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.584651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:43717 (local address 127.8.195.129:57239)
0812 16:52:18.590572 (+ 5921us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:18.590591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:18.590684 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:18.591259 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:18.591265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:18.591331 (+ 66us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:18.591983 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:52:18.591987 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:18.592004 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:52:18.592051 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:18.593040 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:18.593045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:18.593050 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:52:18.593225 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 16:52:18.593279 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5822,"thread_start_us":6900,"threads_started":1}
I20260812 16:52:18.593765 19487 raft_consensus.cc:3037] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:18.594722 19363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e7c3a6c9854d7fb895a3341c2e6217" candidate_uuid: "9e5b33da53584738840adfde581a87b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69488bb83123452799175a47d48b29e2" is_pre_election: true
I20260812 16:52:18.595273 19363 raft_consensus.cc:2454] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b33da53584738840adfde581a87b7 in term 0.
I20260812 16:52:18.596877 19487 raft_consensus.cc:504] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.597620 19487 leader_election.cc:288] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [CANDIDATE]: Term 1 election: Requested vote from peers 2f92ccb77dae4ee19a1554110a01f9c9 (127.8.195.130:41547), 69488bb83123452799175a47d48b29e2 (127.8.195.131:43717)
I20260812 16:52:18.598042 19363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e7c3a6c9854d7fb895a3341c2e6217" candidate_uuid: "9e5b33da53584738840adfde581a87b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69488bb83123452799175a47d48b29e2"
I20260812 16:52:18.598083 19112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e7c3a6c9854d7fb895a3341c2e6217" candidate_uuid: "9e5b33da53584738840adfde581a87b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f92ccb77dae4ee19a1554110a01f9c9"
I20260812 16:52:18.598271 19363 raft_consensus.cc:3037] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:18.598368 19112 raft_consensus.cc:3037] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:18.600782 19363 raft_consensus.cc:2454] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b33da53584738840adfde581a87b7 in term 1.
I20260812 16:52:18.601255 18735 leader_election.cc:302] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69488bb83123452799175a47d48b29e2, 9e5b33da53584738840adfde581a87b7; no voters:
I20260812 16:52:18.601332 19112 raft_consensus.cc:2454] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b33da53584738840adfde581a87b7 in term 1.
I20260812 16:52:18.601603 19487 raft_consensus.cc:2781] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:18.601806 19487 raft_consensus.cc:686] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 1 LEADER]: Becoming Leader. State: Replica: 9e5b33da53584738840adfde581a87b7, State: Running, Role: LEADER
I20260812 16:52:18.602270 19487 consensus_queue.cc:237] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } }
I20260812 16:52:18.606688 18581 catalog_manager.cc:5495] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5b33da53584738840adfde581a87b7 (127.8.195.129). New cstate: current_term: 1 leader_uuid: "9e5b33da53584738840adfde581a87b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5b33da53584738840adfde581a87b7" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 46387 } health_report { overall_health: HEALTHY } } }
I20260812 16:52:18.624778 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.622628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.8.195.129:46387)
0812 16:52:18.622796 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.622818 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.623039 (+ 221us) server_negotiation.cc:407] Connection header received
0812 16:52:18.623145 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.623148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.623264 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.623356 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.624225 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.624230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.624233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.624251 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.624284 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.624295 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.624399 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.624472 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.624518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:52:18.632706 19363 raft_consensus.cc:1261] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 1 FOLLOWER]: Refusing update from remote peer 9e5b33da53584738840adfde581a87b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:18.632714 19112 raft_consensus.cc:1261] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 1 FOLLOWER]: Refusing update from remote peer 9e5b33da53584738840adfde581a87b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:18.633749 19485 consensus_queue.cc:1035] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69488bb83123452799175a47d48b29e2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 43717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:18.634044 19487 consensus_queue.cc:1035] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f92ccb77dae4ee19a1554110a01f9c9" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:18.662482 18581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:18.662786 18581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:18.667301 18581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46cf13872d4248b3bf815faaea06271f: Sending DeleteTablet for 3 replicas of tablet d1e7c3a6c9854d7fb895a3341c2e6217
I20260812 16:52:18.668462 19077 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:18 UTC) from {username='slave'} at 127.0.0.1:37590
I20260812 16:52:18.668540 19322 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:18 UTC) from {username='slave'} at 127.0.0.1:40388
I20260812 16:52:18.670567 18823 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e7c3a6c9854d7fb895a3341c2e6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:18 UTC) from {username='slave'} at 127.0.0.1:54316
I20260812 16:52:18.671293 19525 tablet_replica.cc:330] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: stopping tablet replica
I20260812 16:52:18.671691 19525 raft_consensus.cc:2229] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:18.672152 19525 raft_consensus.cc:2258] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:18.674012 19528 tablet_replica.cc:330] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: stopping tablet replica
I20260812 16:52:18.674247 19525 ts_tablet_manager.cc:1905] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:18.674569 19528 raft_consensus.cc:2229] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:18.675246 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18705
I20260812 16:52:18.675416 19528 raft_consensus.cc:2258] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:18.676230 19526 tablet_replica.cc:330] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: stopping tablet replica
I20260812 16:52:18.676782 19526 raft_consensus.cc:2229] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:18.677368 19526 raft_consensus.cc:2258] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:18.678316 19528 ts_tablet_manager.cc:1905] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:18.679610 19526 ts_tablet_manager.cc:1905] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:18.681532 19525 ts_tablet_manager.cc:1918] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:18.681713 19525 log.cc:1196] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/d1e7c3a6c9854d7fb895a3341c2e6217
I20260812 16:52:18.682271 19525 ts_tablet_manager.cc:1939] T d1e7c3a6c9854d7fb895a3341c2e6217 P 2f92ccb77dae4ee19a1554110a01f9c9: Deleting consensus metadata
I20260812 16:52:18.684140 18558 catalog_manager.cc:4841] TS 2f92ccb77dae4ee19a1554110a01f9c9 (127.8.195.130:41547): tablet d1e7c3a6c9854d7fb895a3341c2e6217 (table test-table [id=67ed73e5c361480394b2281af0584728]) successfully deleted
I20260812 16:52:18.687129 19526 ts_tablet_manager.cc:1918] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:18.687312 19526 log.cc:1196] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/d1e7c3a6c9854d7fb895a3341c2e6217
I20260812 16:52:18.687878 19526 ts_tablet_manager.cc:1939] T d1e7c3a6c9854d7fb895a3341c2e6217 P 69488bb83123452799175a47d48b29e2: Deleting consensus metadata
I20260812 16:52:18.689875 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.677309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.8.195.129:46387)
0812 16:52:18.677475 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.677481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.686897 (+ 9416us) server_negotiation.cc:407] Connection header received
0812 16:52:18.687461 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.687472 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.687550 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.687627 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.688940 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.688946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.688951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.688974 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.689011 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.689026 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.689135 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.689494 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.689576 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:52:18.690469 19528 ts_tablet_manager.cc:1918] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:18.690433 18552 catalog_manager.cc:4841] TS 69488bb83123452799175a47d48b29e2 (127.8.195.131:43717): tablet d1e7c3a6c9854d7fb895a3341c2e6217 (table test-table [id=67ed73e5c361480394b2281af0584728]) successfully deleted
I20260812 16:52:18.690675 19528 log.cc:1196] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/d1e7c3a6c9854d7fb895a3341c2e6217
I20260812 16:52:18.691170 19528 ts_tablet_manager.cc:1939] T d1e7c3a6c9854d7fb895a3341c2e6217 P 9e5b33da53584738840adfde581a87b7: Deleting consensus metadata
I20260812 16:52:18.691581 18946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:18.790495 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18705
I20260812 16:52:18.790715 18552 catalog_manager.cc:4841] TS 9e5b33da53584738840adfde581a87b7 (127.8.195.129:46387): tablet d1e7c3a6c9854d7fb895a3341c2e6217 (table test-table [id=67ed73e5c361480394b2281af0584728]) successfully deleted
I20260812 16:52:18.818431 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18963
I20260812 16:52:18.821329 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.819355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.8.195.130:41547)
0812 16:52:18.819529 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.819534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.819552 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:52:18.819609 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.819612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.819705 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.819792 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.820728 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.820732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.820735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.820749 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.820777 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.820788 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.820886 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.820982 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.821027 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:52:18.822073 19185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:18.910846 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18963
I20260812 16:52:18.940572 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19200
I20260812 16:52:18.943940 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:18.941593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.8.195.131:43717)
0812 16:52:18.941794 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:18.941800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:18.941823 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:18.941984 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:18.941994 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:18.942076 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:18.942188 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:18.943214 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:18.943220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:18.943225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:18.943246 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:18.943287 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:18.943300 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:18.943462 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:18.943563 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 16:52:18.943629 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:52:18.944700 19426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:19.035912 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19200
I20260812 16:52:19.062039 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18521
I20260812 16:52:19.063274 18645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:19.171497 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18521
2026-08-12T16:52:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2701 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 16:52:19.223489 8974 mini_oidc.cc:152] Starting JWKS server
I20260812 16:52:19.226553 8974 webserver.cc:460] Webserver started at https://127.0.0.1:33269/ using document root <none> and password file <none>
I20260812 16:52:19.226917 8974 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:52:19.227620 8974 webserver.cc:460] Webserver started at http://127.0.0.1:43077/ using document root <none> and password file <none>
2026-08-12T16:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:19Z Disabled control of system clock
I20260812 16:52:19.262445 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37509
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37509
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33269/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/testchainca.cert with env {}
W20260812 16:52:19.592522 19558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:19.592872 19558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:19.593043 19558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:19.600369 19558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:19.600522 19558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:19.600622 19558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:19.600718 19558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:19.600803 19558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:52:19.600894 19558 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:52:19.600977 19558 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33269/jwks.json
I20260812 16:52:19.611061 19558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33269/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:19.612423 19558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:19.613759 19558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:19.625523 19568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:19.625502 19575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:19.625566 19570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:19.626477 19558 server_base.cc:1034] running on GCE node
I20260812 16:52:19.627266 19558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:19.628738 19558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:19.629972 19558 hybrid_clock.cc:648] HybridClock initialized: now 1786553539629938 us; error 54 us; skew 500 ppm
I20260812 16:52:19.630337 19558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:19.632647 19558 webserver.cc:460] Webserver started at http://127.8.195.190:40659/ using document root <none> and password file <none>
I20260812 16:52:19.633425 19558 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:19.633569 19558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:19.633900 19558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:19.636804 19558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "ffc47e8513914b9291afb739d841d9a1"
format_stamp: "Formatted at 2026-08-12 16:52:19 on dist-test-slave-z8x9"
server_key: "7948d1e51b10a67646e4031af19ee465"
server_key_iv: "f9119c6f20eedbf37370284ba76e3afb"
server_key_version: "encryptionkey@0"
I20260812 16:52:19.637676 19558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "ffc47e8513914b9291afb739d841d9a1"
format_stamp: "Formatted at 2026-08-12 16:52:19 on dist-test-slave-z8x9"
server_key: "7948d1e51b10a67646e4031af19ee465"
server_key_iv: "f9119c6f20eedbf37370284ba76e3afb"
server_key_version: "encryptionkey@0"
I20260812 16:52:19.642901 19558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:19.646499 19582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:19.647915 19558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:19.648108 19558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "ffc47e8513914b9291afb739d841d9a1"
format_stamp: "Formatted at 2026-08-12 16:52:19 on dist-test-slave-z8x9"
server_key: "7948d1e51b10a67646e4031af19ee465"
server_key_iv: "f9119c6f20eedbf37370284ba76e3afb"
server_key_version: "encryptionkey@0"
I20260812 16:52:19.648347 19558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:19.672899 19558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:19.673830 19558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:19.674172 19558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:19.693420 19558 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37509
I20260812 16:52:19.693367 19685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37509 every 8 connection(s)
I20260812 16:52:19.694766 19558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:19.698606 19690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:19.703430 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19558
I20260812 16:52:19.703662 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:19.704312 8974 external_mini_cluster.cc:1428] Setting key 5362fbcf313a8c5c6cce2930dbb4ce4f
I20260812 16:52:19.709851 19690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1: Bootstrap starting.
I20260812 16:52:19.713613 19690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:19.714830 19690 log.cc:824] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:19.718144 19690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1: No bootstrap required, opened a new log
I20260812 16:52:19.718709 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:19.706260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.8.195.190:37509)
0812 16:52:19.707341 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:19.707352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:19.707395 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:52:19.709101 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:19.709125 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:19.709652 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:19.710002 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:19.710703 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.712033 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.714024 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.714699 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.714841 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:19.716245 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:19.716257 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:19.716280 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:19.716829 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:19.716906 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:19.716926 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:19.717024 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:19.717360 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 16:52:19.717709 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":392,"threads_started":1}
I20260812 16:52:19.722579 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:52:19.727061 19690 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } }
I20260812 16:52:19.727401 19690 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:19.727474 19690 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc47e8513914b9291afb739d841d9a1, State: Initialized, Role: FOLLOWER
I20260812 16:52:19.727955 19690 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } }
I20260812 16:52:19.728189 19690 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:19.728329 19690 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:19.728505 19690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:19.731392 19690 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } }
I20260812 16:52:19.731848 19690 leader_election.cc:302] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc47e8513914b9291afb739d841d9a1; no voters:
I20260812 16:52:19.732499 19690 leader_election.cc:288] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:19.732651 19697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:19.733539 19697 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [term 1 LEADER]: Becoming Leader. State: Replica: ffc47e8513914b9291afb739d841d9a1, State: Running, Role: LEADER
I20260812 16:52:19.734071 19697 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } }
I20260812 16:52:19.734838 19690 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:19.737649 19699 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc47e8513914b9291afb739d841d9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } } }
I20260812 16:52:19.737639 19700 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc47e8513914b9291afb739d841d9a1. Latest consensus state: current_term: 1 leader_uuid: "ffc47e8513914b9291afb739d841d9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc47e8513914b9291afb739d841d9a1" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37509 } } }
I20260812 16:52:19.738140 19699 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:19.738211 19700 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:19.742023 19710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:19.747835 19710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:19.757818 19710 catalog_manager.cc:1347] Generated new cluster ID: ff8fd1dbcf004115b77f74909ae46e5c
I20260812 16:52:19.757961 19710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:19.839249 19710 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:19.840579 19710 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:19.852622 19710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc47e8513914b9291afb739d841d9a1: Generated new TSK 0
I20260812 16:52:19.853725 19710 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:52:19.899597 19748 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 16:52:19.899842 19748 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:52:19.901101 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:19.878250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.8.195.190:37509)
0812 16:52:19.878470 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:19.878478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:19.878985 (+ 507us) server_negotiation.cc:407] Connection header received
0812 16:52:19.879352 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:19.879358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:19.879449 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:19.879624 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:52:19.880142 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.881317 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.882716 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.883399 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.883532 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:19.884066 (+ 534us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:52:19.900388 (+ 16322us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:52:19.900784 (+ 396us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 16:52:19.901265 19695 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 16:52:19.902747 19747 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:19.878040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37509 (local address 127.0.0.1:59388)
0812 16:52:19.878880 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:19.878898 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:19.879052 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:19.879665 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:19.879670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:19.879680 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:52:19.879991 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:19.879999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.881465 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.881469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:19.882558 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:19.882570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:19.883575 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:19.883579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:19.883713 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:19.883774 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:52:19.900715 (+ 16941us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 16:52:19.900989 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.195.190:37509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":474,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.195.190:37509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 16:52:19.904531 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19558
I20260812 16:52:19.905810 19665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:20.011229 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19558
2026-08-12T16:52:20Z chronyd exiting
I20260812 16:52:20.071768 8974 test_util.cc:175] -----------------------------------------------
I20260812 16:52:20.071921 8974 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786553491558846-8974-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (853 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 16:52:20.075879 8974 mini_oidc.cc:152] Starting JWKS server
I20260812 16:52:20.077817 8974 webserver.cc:460] Webserver started at https://127.0.0.1:37159/ using document root <none> and password file <none>
I20260812 16:52:20.078020 8974 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:52:20.078624 8974 webserver.cc:460] Webserver started at http://127.0.0.1:41649/ using document root <none> and password file <none>
2026-08-12T16:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:20Z Disabled control of system clock
I20260812 16:52:20.117725 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45201
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45201
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37159/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/testchainca.cert with env {}
W20260812 16:52:20.458091 19760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:20.458454 19760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:20.458678 19760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:20.466336 19760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:20.466463 19760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:20.466558 19760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:20.466668 19760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:20.466751 19760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:52:20.466847 19760 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:52:20.466907 19760 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37159/jwks.json
I20260812 16:52:20.477294 19760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37159/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:20.478770 19760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:20.480131 19760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:20.492964 19771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:20.493057 19775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:20.492990 19772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:20.493683 19760 server_base.cc:1034] running on GCE node
I20260812 16:52:20.494585 19760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:20.496083 19760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:20.497272 19760 hybrid_clock.cc:648] HybridClock initialized: now 1786553540497231 us; error 55 us; skew 500 ppm
I20260812 16:52:20.497644 19760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:20.505097 19760 webserver.cc:460] Webserver started at http://127.8.195.190:37053/ using document root <none> and password file <none>
I20260812 16:52:20.505851 19760 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:20.505991 19760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:20.506327 19760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:20.509299 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "1c0e9fe7d0d540af9097879034271eff"
format_stamp: "Formatted at 2026-08-12 16:52:20 on dist-test-slave-z8x9"
server_key: "49442b9aa3d2b328a8122510cc42ab74"
server_key_iv: "269b37561c7336de9a62acbb6e7f77bc"
server_key_version: "encryptionkey@0"
I20260812 16:52:20.510123 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "1c0e9fe7d0d540af9097879034271eff"
format_stamp: "Formatted at 2026-08-12 16:52:20 on dist-test-slave-z8x9"
server_key: "49442b9aa3d2b328a8122510cc42ab74"
server_key_iv: "269b37561c7336de9a62acbb6e7f77bc"
server_key_version: "encryptionkey@0"
I20260812 16:52:20.515278 19760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260812 16:52:20.518867 19783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:20.520272 19760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 16:52:20.520478 19760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "1c0e9fe7d0d540af9097879034271eff"
format_stamp: "Formatted at 2026-08-12 16:52:20 on dist-test-slave-z8x9"
server_key: "49442b9aa3d2b328a8122510cc42ab74"
server_key_iv: "269b37561c7336de9a62acbb6e7f77bc"
server_key_version: "encryptionkey@0"
I20260812 16:52:20.520707 19760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:20.542508 19760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:20.543381 19760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:20.543713 19760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:20.562713 19760 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:45201
I20260812 16:52:20.562719 19892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:45201 every 8 connection(s)
I20260812 16:52:20.564173 19760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:20.567971 19894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:20.573704 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19760
I20260812 16:52:20.573920 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:20.574577 8974 external_mini_cluster.cc:1428] Setting key 636e01b089f8990282380f3ae668815e
I20260812 16:52:20.579128 19894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff: Bootstrap starting.
I20260812 16:52:20.582885 19894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:20.584169 19894 log.cc:824] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:20.587548 19894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff: No bootstrap required, opened a new log
I20260812 16:52:20.589272 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:20.576691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.8.195.190:45201)
0812 16:52:20.577781 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:20.577792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:20.577834 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:52:20.579535 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:20.579559 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:20.579996 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:20.580300 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:20.580998 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.582381 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.584303 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.585380 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.585567 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:20.586839 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:20.586846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:20.586860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:20.587436 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:20.587517 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:20.587541 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:20.587650 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:20.587884 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 16:52:20.588268 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":422,"threads_started":1}
I20260812 16:52:20.593510 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:52:20.596501 19894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } }
I20260812 16:52:20.597023 19894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:20.597154 19894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0e9fe7d0d540af9097879034271eff, State: Initialized, Role: FOLLOWER
I20260812 16:52:20.597735 19894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } }
I20260812 16:52:20.597961 19894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:20.598061 19894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:20.598187 19894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:20.600502 19894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } }
I20260812 16:52:20.601194 19894 leader_election.cc:302] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c0e9fe7d0d540af9097879034271eff; no voters:
I20260812 16:52:20.601857 19894 leader_election.cc:288] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:20.602001 19901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:20.602761 19901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [term 1 LEADER]: Becoming Leader. State: Replica: 1c0e9fe7d0d540af9097879034271eff, State: Running, Role: LEADER
I20260812 16:52:20.603327 19901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } }
I20260812 16:52:20.604023 19894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:20.606088 19904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0e9fe7d0d540af9097879034271eff. Latest consensus state: current_term: 1 leader_uuid: "1c0e9fe7d0d540af9097879034271eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } } }
I20260812 16:52:20.606477 19904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:20.607263 19903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c0e9fe7d0d540af9097879034271eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0e9fe7d0d540af9097879034271eff" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45201 } } }
I20260812 16:52:20.607703 19903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:20.608109 19911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:20.613353 19911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:20.622728 19911 catalog_manager.cc:1347] Generated new cluster ID: cd8e30abdd104e60b6a53f9513a205c0
I20260812 16:52:20.622857 19911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:20.637997 19911 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:20.639364 19911 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:20.645687 19911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c0e9fe7d0d540af9097879034271eff: Generated new TSK 0
I20260812 16:52:20.646544 19911 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 16:52:20.694217 19954 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 16:52:20.694365 19954 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:52:20.695681 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:20.674488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.8.195.190:45201)
0812 16:52:20.674694 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:20.674701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:20.675182 (+ 481us) server_negotiation.cc:407] Connection header received
0812 16:52:20.675410 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:20.675416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:20.675517 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:20.675708 (+ 191us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:52:20.676157 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.677404 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.678870 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.679693 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.679854 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:20.680400 (+ 546us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:52:20.694974 (+ 14574us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37159/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:52:20.695394 (+ 420us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37159/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 16:52:20.695744 19951 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:20.674276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45201 (local address 127.0.0.1:45348)
0812 16:52:20.675066 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:20.675082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:20.675218 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:20.675720 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:20.675724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:20.675733 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:52:20.675997 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:20.676006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.677545 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.677548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:20.678723 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:20.678737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:20.679922 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:20.679928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:20.680073 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:20.680107 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:52:20.695231 (+ 15124us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37159/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 16:52:20.695460 (+ 229us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.195.190:45201: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37159/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":471,"threads_started":1}
W20260812 16:52:20.695825 19897 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37159/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 16:52:20.697335 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19760
I20260812 16:52:20.698597 19885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:20.793756 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19760
2026-08-12T16:52:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (786 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 16:52:20.862528 8974 mini_oidc.cc:152] Starting JWKS server
I20260812 16:52:20.864391 8974 webserver.cc:460] Webserver started at https://127.0.0.1:43009/ using document root <none> and password file <none>
I20260812 16:52:20.864598 8974 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 16:52:20.865217 8974 webserver.cc:460] Webserver started at http://127.0.0.1:36705/ using document root <none> and password file <none>
2026-08-12T16:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:20Z Disabled control of system clock
I20260812 16:52:20.901396 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34087
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34087
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43009/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:52:21.243396 19966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:21.243712 19966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:21.243893 19966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:21.251451 19966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:21.251577 19966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:21.251650 19966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:21.251744 19966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:21.251852 19966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:52:21.251960 19966 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:52:21.252019 19966 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43009/jwks.json
I20260812 16:52:21.261835 19966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43009/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:21.263242 19966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:21.265251 19966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:21.277608 19977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:21.277639 19984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:21.277549 19979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:21.278612 19966 server_base.cc:1034] running on GCE node
I20260812 16:52:21.279548 19966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:21.281046 19966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:21.282286 19966 hybrid_clock.cc:648] HybridClock initialized: now 1786553541282240 us; error 62 us; skew 500 ppm
I20260812 16:52:21.282683 19966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:21.285274 19966 webserver.cc:460] Webserver started at http://127.8.195.190:38871/ using document root <none> and password file <none>
I20260812 16:52:21.286118 19966 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:21.286259 19966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:21.286600 19966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:21.289649 19966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "ba5b0e94a0b14139a236f2cf46a1460c"
format_stamp: "Formatted at 2026-08-12 16:52:21 on dist-test-slave-z8x9"
server_key: "d504b06039bbb5334968ee683463e7a3"
server_key_iv: "c8d79c0265d04f81d622a0bb556ec516"
server_key_version: "encryptionkey@0"
I20260812 16:52:21.290495 19966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "ba5b0e94a0b14139a236f2cf46a1460c"
format_stamp: "Formatted at 2026-08-12 16:52:21 on dist-test-slave-z8x9"
server_key: "d504b06039bbb5334968ee683463e7a3"
server_key_iv: "c8d79c0265d04f81d622a0bb556ec516"
server_key_version: "encryptionkey@0"
I20260812 16:52:21.295740 19966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:21.299329 19996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:21.300769 19966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:21.300979 19966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "ba5b0e94a0b14139a236f2cf46a1460c"
format_stamp: "Formatted at 2026-08-12 16:52:21 on dist-test-slave-z8x9"
server_key: "d504b06039bbb5334968ee683463e7a3"
server_key_iv: "c8d79c0265d04f81d622a0bb556ec516"
server_key_version: "encryptionkey@0"
I20260812 16:52:21.301231 19966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:21.321403 19966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:21.322314 19966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:21.322651 19966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:21.342056 19966 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34087
I20260812 16:52:21.342079 20086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34087 every 8 connection(s)
I20260812 16:52:21.343482 19966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:21.347278 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19966
I20260812 16:52:21.347489 20087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:21.347500 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:21.348167 8974 external_mini_cluster.cc:1428] Setting key ff2e9a4a13919f196342c4421e49cd89
I20260812 16:52:21.362550 20087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c: Bootstrap starting.
I20260812 16:52:21.364769 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:21.350347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.8.195.190:34087)
0812 16:52:21.351489 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:21.351504 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:21.351559 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:52:21.354346 (+ 2787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:21.354376 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:21.355021 (+ 645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:21.355474 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:21.355997 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.357922 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.359283 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.360406 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.360576 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:21.361871 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:21.361882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:21.361903 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:21.362649 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:21.362754 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:21.362780 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:21.362907 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:21.363146 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 16:52:21.363464 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":356,"threads_started":1}
I20260812 16:52:21.366482 20087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:21.367740 20087 log.cc:824] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:21.368839 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:52:21.370816 20087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c: No bootstrap required, opened a new log
I20260812 16:52:21.378746 20087 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } }
I20260812 16:52:21.379133 20087 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:21.379261 20087 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba5b0e94a0b14139a236f2cf46a1460c, State: Initialized, Role: FOLLOWER
I20260812 16:52:21.379882 20087 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } }
I20260812 16:52:21.380095 20087 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:21.380283 20087 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:21.380475 20087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:21.382779 20087 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } }
I20260812 16:52:21.383440 20087 leader_election.cc:302] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba5b0e94a0b14139a236f2cf46a1460c; no voters:
I20260812 16:52:21.384090 20087 leader_election.cc:288] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:21.384370 20095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:21.385226 20095 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [term 1 LEADER]: Becoming Leader. State: Replica: ba5b0e94a0b14139a236f2cf46a1460c, State: Running, Role: LEADER
I20260812 16:52:21.385732 20095 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } }
I20260812 16:52:21.386323 20087 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:21.388841 20096 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } } }
I20260812 16:52:21.389325 20096 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:21.390861 20097 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba5b0e94a0b14139a236f2cf46a1460c. Latest consensus state: current_term: 1 leader_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5b0e94a0b14139a236f2cf46a1460c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34087 } } }
I20260812 16:52:21.391355 20097 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:21.394179 20112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:21.399919 20112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:21.409582 20112 catalog_manager.cc:1347] Generated new cluster ID: 502443cb47114a219ad9f799c47ab56a
I20260812 16:52:21.409729 20112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:21.432449 20112 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:21.434398 20112 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:21.446480 20112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba5b0e94a0b14139a236f2cf46a1460c: Generated new TSK 0
I20260812 16:52:21.447367 20112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:52:21.484321 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:21.474855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.8.195.190:34087)
0812 16:52:21.475092 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:21.475099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:21.475603 (+ 504us) server_negotiation.cc:407] Connection header received
0812 16:52:21.475851 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:21.475857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:21.475967 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:21.476169 (+ 202us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:52:21.476629 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.477883 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.479295 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.480010 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.480142 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:21.480712 (+ 570us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:52:21.483325 (+ 2613us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43009/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:52:21.483985 (+ 660us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54266: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43009/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 16:52:21.484467 20092 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54266: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43009/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 16:52:21.485777 20158 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:21.474689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34087 (local address 127.0.0.1:54266)
0812 16:52:21.475475 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:21.475493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:21.475640 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:21.476174 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:21.476178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:21.476188 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:52:21.476469 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:21.476478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.478040 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.478044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:21.479141 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:21.479154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:21.480212 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:21.480217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:21.480366 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:21.480404 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:52:21.483587 (+ 3183us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43009/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 16:52:21.485267 (+ 1680us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.195.190:34087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43009/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":1287,"thread_start_us":453,"threads_started":1}
I20260812 16:52:21.488303 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19966
I20260812 16:52:21.489735 20078 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 16:52:21.495685 20164 webserver.cc:533] Webserver: sslize failed:
E20260812 16:52:21.495833 20164 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 16:52:21.585991 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19966
2026-08-12T16:52:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (788 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T16:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:21Z Disabled control of system clock
I20260812 16:52:21.690896 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:43349
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:43349
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/testchainca.cert with env {}
W20260812 16:52:22.037038 20173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:22.037468 20173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:22.037563 20173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:22.044862 20173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:22.044981 20173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:22.045019 20173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:22.045079 20173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:22.045125 20173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 16:52:22.045274 20173 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 16:52:22.045348 20173 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 16:52:22.055922 20173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:43349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:43349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:22.057474 20173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:22.058835 20173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:22.071126 20183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:22.071048 20187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:22.071092 20185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:22.072021 20173 server_base.cc:1034] running on GCE node
I20260812 16:52:22.072893 20173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:22.074358 20173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:22.075567 20173 hybrid_clock.cc:648] HybridClock initialized: now 1786553542075526 us; error 28 us; skew 500 ppm
I20260812 16:52:22.075943 20173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:52:22.078400 20173 webserver.cc:460] Webserver started at http://127.8.195.190:36911/ using document root <none> and password file <none>
I20260812 16:52:22.079097 20173 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:22.079224 20173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:22.079571 20173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:22.082531 20173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "c881a65178ef44b39b907dc2867cb01d"
format_stamp: "Formatted at 2026-08-12 16:52:22 on dist-test-slave-z8x9"
server_key: "a540b50bb92906b03f11ca5af3234366"
server_key_iv: "dfbab4f45a47bff04f36a5031951b7cb"
server_key_version: "encryptionkey@0"
I20260812 16:52:22.083336 20173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "c881a65178ef44b39b907dc2867cb01d"
format_stamp: "Formatted at 2026-08-12 16:52:22 on dist-test-slave-z8x9"
server_key: "a540b50bb92906b03f11ca5af3234366"
server_key_iv: "dfbab4f45a47bff04f36a5031951b7cb"
server_key_version: "encryptionkey@0"
I20260812 16:52:22.088450 20173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 16:52:22.091966 20200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:22.093325 20173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:22.093513 20173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "c881a65178ef44b39b907dc2867cb01d"
format_stamp: "Formatted at 2026-08-12 16:52:22 on dist-test-slave-z8x9"
server_key: "a540b50bb92906b03f11ca5af3234366"
server_key_iv: "dfbab4f45a47bff04f36a5031951b7cb"
server_key_version: "encryptionkey@0"
I20260812 16:52:22.093735 20173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:22.108551 20173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:22.109543 20173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:22.109840 20173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:22.130142 20173 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:43349
I20260812 16:52:22.130162 20301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:43349 every 8 connection(s)
I20260812 16:52:22.131575 20173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:22.135792 20303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:22.137491 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20173
I20260812 16:52:22.137696 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:22.138399 8974 external_mini_cluster.cc:1428] Setting key 8f6a9f2193032c9a153be070d909694c
I20260812 16:52:22.149469 20303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d: Bootstrap starting.
I20260812 16:52:22.153402 20303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:22.154556 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:22.140413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.8.195.190:43349)
0812 16:52:22.141537 (+ 1124us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:22.141552 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:22.141601 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:52:22.143964 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:22.143991 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:22.144552 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:22.144996 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:22.145579 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.147649 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.148957 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.150058 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.150240 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:22.151523 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:22.151534 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:22.151558 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:52:22.152314 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:22.152453 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:52:22.152485 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:22.152603 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:22.152844 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 16:52:22.153201 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":330,"threads_started":1}
I20260812 16:52:22.154765 20303 log.cc:824] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:22.158337 20303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d: No bootstrap required, opened a new log
I20260812 16:52:22.159291 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:52:22.166950 20303 raft_consensus.cc:348] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } }
I20260812 16:52:22.167354 20303 raft_consensus.cc:374] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:22.167466 20303 raft_consensus.cc:729] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c881a65178ef44b39b907dc2867cb01d, State: Initialized, Role: FOLLOWER
I20260812 16:52:22.168030 20303 consensus_queue.cc:260] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } }
I20260812 16:52:22.168272 20303 raft_consensus.cc:388] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:22.168429 20303 raft_consensus.cc:482] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:22.168620 20303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:22.171255 20303 raft_consensus.cc:504] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } }
I20260812 16:52:22.171813 20303 leader_election.cc:302] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c881a65178ef44b39b907dc2867cb01d; no voters:
I20260812 16:52:22.172461 20303 leader_election.cc:288] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:22.172622 20311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:22.173386 20311 raft_consensus.cc:686] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [term 1 LEADER]: Becoming Leader. State: Replica: c881a65178ef44b39b907dc2867cb01d, State: Running, Role: LEADER
I20260812 16:52:22.173900 20311 consensus_queue.cc:237] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } }
I20260812 16:52:22.174618 20303 sys_catalog.cc:564] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:22.176822 20313 sys_catalog.cc:455] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c881a65178ef44b39b907dc2867cb01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } } }
I20260812 16:52:22.177378 20313 sys_catalog.cc:458] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:22.177409 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c881a65178ef44b39b907dc2867cb01d. Latest consensus state: current_term: 1 leader_uuid: "c881a65178ef44b39b907dc2867cb01d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c881a65178ef44b39b907dc2867cb01d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 43349 } } }
I20260812 16:52:22.177786 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:22.182862 20324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:22.188493 20324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:22.198040 20324 catalog_manager.cc:1347] Generated new cluster ID: 22882889dad144a5a0cbb27daa47cd56
I20260812 16:52:22.198177 20324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:22.224901 20324 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:22.226891 20324 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:22.241747 20324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c881a65178ef44b39b907dc2867cb01d: Generated new TSK 0
I20260812 16:52:22.242863 20324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:52:22.334964 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:22.261675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.8.195.190:43349)
0812 16:52:22.261935 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:22.261946 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:22.262426 (+ 480us) server_negotiation.cc:407] Connection header received
0812 16:52:22.262669 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:22.262673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:22.262770 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:22.262909 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0812 16:52:22.263493 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.264256 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.266405 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.266907 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.267012 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:22.267631 (+ 619us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 16:52:22.334353 (+ 66722us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:52:22.334706 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38098: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 16:52:22.335155 20309 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38098: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 16:52:22.335443 20363 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:22.261642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:43349 (local address 127.0.0.1:38098)
0812 16:52:22.262307 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:22.262329 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:22.262488 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:22.262954 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:22.262960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:22.262973 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0812 16:52:22.263310 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:22.263322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.264423 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.264429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:22.266244 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:22.266260 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:22.267091 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:22.267098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:22.267313 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:22.267361 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 16:52:22.334677 (+ 67316us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 16:52:22.335006 (+ 329us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.195.190:43349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":329,"threads_started":1}
I20260812 16:52:22.337515 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20173
I20260812 16:52:22.338933 20291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:22.442552 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20173
2026-08-12T16:52:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (829 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (449 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1575 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:24 dist-test-slave-z8x9 krb5kdc[20395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:24 dist-test-slave-z8x9 krb5kdc[20395](info): set up 2 sockets
Aug 12 16:52:24 dist-test-slave-z8x9 krb5kdc[20395](info): commencing operation
krb5kdc: starting...
W20260812 16:52:25.863962 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:26 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:26.129345 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37609
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:26.483117 20416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:26.483435 20416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:26.483530 20416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:26.490904 20416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:26.491027 20416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:26.491074 20416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:26.491132 20416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:26.491178 20416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:26.501315 20416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:26.502642 20416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:26.503917 20416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:26.516281 20428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:26.516328 20436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:26.516393 20440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:26.522871 20416 server_base.cc:1034] running on GCE node
I20260812 16:52:26.523764 20416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:26.525282 20416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:26.526549 20416 hybrid_clock.cc:648] HybridClock initialized: now 1786553546526516 us; error 54 us; skew 500 ppm
I20260812 16:52:26.526934 20416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:26 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553546, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:26.534710 20416 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:26.536535 20416 webserver.cc:460] Webserver started at http://127.8.195.190:41089/ using document root <none> and password file <none>
I20260812 16:52:26.537477 20416 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:26.537639 20416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:26.537988 20416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:26.541144 20416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "c6c491b769eb42b6b9afe8b800e1991c"
format_stamp: "Formatted at 2026-08-12 16:52:26 on dist-test-slave-z8x9"
server_key: "e8a5383f974f4551e3447ab4d04fbaa7"
server_key_iv: "f1d09b79bb16fa8b912508edda374913"
server_key_version: "encryptionkey@0"
I20260812 16:52:26.542085 20416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "c6c491b769eb42b6b9afe8b800e1991c"
format_stamp: "Formatted at 2026-08-12 16:52:26 on dist-test-slave-z8x9"
server_key: "e8a5383f974f4551e3447ab4d04fbaa7"
server_key_iv: "f1d09b79bb16fa8b912508edda374913"
server_key_version: "encryptionkey@0"
I20260812 16:52:26.547488 20416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 16:52:26.551141 20450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:26.552587 20416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 16:52:26.552783 20416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "c6c491b769eb42b6b9afe8b800e1991c"
format_stamp: "Formatted at 2026-08-12 16:52:26 on dist-test-slave-z8x9"
server_key: "e8a5383f974f4551e3447ab4d04fbaa7"
server_key_iv: "f1d09b79bb16fa8b912508edda374913"
server_key_version: "encryptionkey@0"
I20260812 16:52:26.553001 20416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:26.572088 20416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:26.576165 20416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:26.576479 20416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:26.595369 20416 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37609
I20260812 16:52:26.595410 20547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37609 every 8 connection(s)
I20260812 16:52:26.596787 20416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:26.600634 20549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:26.607203 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20416
I20260812 16:52:26.607445 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:26.608320 8974 external_mini_cluster.cc:1428] Setting key c28f1215bd656f7bc96e509efa65908d
I20260812 16:52:26.612365 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c: Bootstrap starting.
I20260812 16:52:26.615548 20549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:26.616797 20549 log.cc:824] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:26.619899 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c: No bootstrap required, opened a new log
Aug 12 16:52:26 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:26.629005 20549 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } }
I20260812 16:52:26.629504 20549 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:26.629624 20549 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c491b769eb42b6b9afe8b800e1991c, State: Initialized, Role: FOLLOWER
I20260812 16:52:26.630335 20549 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } }
I20260812 16:52:26.630631 20549 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:26.630815 20549 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:26.631024 20549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:26.633067 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:26.610598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.8.195.190:37609)
0812 16:52:26.611437 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:26.611449 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:26.611490 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:52:26.613426 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:26.613448 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:26.613832 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:26.614240 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:26.615734 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:26.617148 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:26.618900 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:26.619578 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:26.619715 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:26.624618 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:26.624628 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:26.624647 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:26.624697 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:26.628139 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:26.628895 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:26.628902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:26.628909 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:26.629076 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:26.629515 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:26.629524 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:26.629530 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:26.630276 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:26.630605 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:26.631126 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 16:52:26.631577 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":386,"threads_started":1}
I20260812 16:52:26.634424 20549 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } }
I20260812 16:52:26.634982 20549 leader_election.cc:302] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6c491b769eb42b6b9afe8b800e1991c; no voters:
I20260812 16:52:26.635609 20549 leader_election.cc:288] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:26.635784 20559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:26.636874 20559 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [term 1 LEADER]: Becoming Leader. State: Replica: c6c491b769eb42b6b9afe8b800e1991c, State: Running, Role: LEADER
I20260812 16:52:26.637463 20559 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } }
I20260812 16:52:26.637807 20549 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:26.640326 20564 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6c491b769eb42b6b9afe8b800e1991c. Latest consensus state: current_term: 1 leader_uuid: "c6c491b769eb42b6b9afe8b800e1991c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } } }
I20260812 16:52:26.640693 20564 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:26.642033 20574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:26.641845 20561 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6c491b769eb42b6b9afe8b800e1991c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6c491b769eb42b6b9afe8b800e1991c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37609 } } }
I20260812 16:52:26.642241 20561 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:26.648177 20574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:26.661077 20574 catalog_manager.cc:1347] Generated new cluster ID: c89b79a113964690b79286fc059b7e30
I20260812 16:52:26.661243 20574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:26.686169 20574 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:26.687641 20574 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:26.697540 20574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6c491b769eb42b6b9afe8b800e1991c: Generated new TSK 0
I20260812 16:52:26.698663 20574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:52:26.764940 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:27.122694 20594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:27.123036 20594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:27.123208 20594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:27.130669 20594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:27.130884 20594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:27.131067 20594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:27.143162 20594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:27.144527 20594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:27.145960 20594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:27.159741 20604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:27.159703 20602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:27.159801 20601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:27.160753 20594 server_base.cc:1034] running on GCE node
I20260812 16:52:27.161487 20594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:27.162501 20594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:27.163750 20594 hybrid_clock.cc:648] HybridClock initialized: now 1786553547163679 us; error 58 us; skew 500 ppm
I20260812 16:52:27.164083 20594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:27 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:27.167501 20594 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:52:27.169067 20594 webserver.cc:460] Webserver started at http://127.8.195.129:41163/ using document root <none> and password file <none>
I20260812 16:52:27.169814 20594 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:27.169953 20594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:27.170276 20594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:27.173220 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "5498094a20cd4700bdb9b64be7a7ac9e"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "05cddbde48bb18c8b875ec60a65bede2"
server_key_iv: "d3160b978d2ea3370e4ef396fcca665f"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.174027 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "5498094a20cd4700bdb9b64be7a7ac9e"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "05cddbde48bb18c8b875ec60a65bede2"
server_key_iv: "d3160b978d2ea3370e4ef396fcca665f"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.179062 20594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:27.182600 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.183879 20594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:27.184044 20594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "5498094a20cd4700bdb9b64be7a7ac9e"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "05cddbde48bb18c8b875ec60a65bede2"
server_key_iv: "d3160b978d2ea3370e4ef396fcca665f"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.184266 20594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:27.209596 20594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:27.214471 20594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:27.214809 20594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:27.215862 20594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:27.217334 20594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:27.217410 20594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.217576 20594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:27.217638 20594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.252494 20594 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:46445
I20260812 16:52:27.252645 20886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:46445 every 8 connection(s)
I20260812 16:52:27.253917 20594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:27.264076 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20594
I20260812 16:52:27.264303 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:27.265200 8974 external_mini_cluster.cc:1428] Setting key 2fe7f1f4629132e2925fc64a8c71c7c8
Aug 12 16:52:27 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:27.276589 20889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:27.256790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:55367 (local address 127.8.195.190:37609)
0812 16:52:27.257361 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:27.257368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:27.258528 (+ 1160us) server_negotiation.cc:407] Connection header received
0812 16:52:27.260308 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:27.260313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:27.260387 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:27.260506 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:27.262119 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.262896 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.264570 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.265273 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.265429 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:27.269564 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:27.269569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:27.269572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:27.269607 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:27.272870 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:27.273874 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:27.273880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:27.273882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:27.274022 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:27.274575 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:27.274580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:27.274582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:27.274890 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:27.275074 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:27.275907 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 16:52:27.276173 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":224,"threads_started":1}
I20260812 16:52:27.277812 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:27.257504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37609 (local address 127.8.195.129:55367)
0812 16:52:27.258346 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:27.258398 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:27.260038 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:27.260757 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:27.260765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:27.261373 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:27.261946 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:27.261964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.263067 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.263071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:27.264410 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:27.264426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.265530 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.265536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:27.265765 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:27.266699 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:27.266726 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:27.269330 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:27.273077 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:27.273087 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:27.273114 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:27.273705 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:27.274183 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:27.274188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:27.274192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:27.274423 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:27.275055 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:27.275065 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:27.275465 (+ 400us) client_negotiation.cc:769] Sending connection context
0812 16:52:27.275771 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 16:52:27.276431 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":318,"threads_started":1}
I20260812 16:52:27.279537 20887 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37609
I20260812 16:52:27.279904 20887 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:27.280872 20887 heartbeater.cc:511] Master 127.8.195.190:37609 requested a full tablet report, sending...
I20260812 16:52:27.283324 20479 ts_manager.cc:194] Registered new tserver with Master: 5498094a20cd4700bdb9b64be7a7ac9e (127.8.195.129:46445)
I20260812 16:52:27.284837 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:55367
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:52:27.324901 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:27.662899 20903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:27.663177 20903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:27.663275 20903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:27.670600 20903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:27.670704 20903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:27.670817 20903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:27.680613 20903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:27.681984 20903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:27.683239 20903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:27.697860 20915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:27.697731 20919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:27.697743 20916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:27.698604 20903 server_base.cc:1034] running on GCE node
I20260812 16:52:27.699292 20903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:27.700240 20903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:27.701498 20903 hybrid_clock.cc:648] HybridClock initialized: now 1786553547701450 us; error 39 us; skew 500 ppm
I20260812 16:52:27.701850 20903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:27 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:27.705438 20903 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:52:27.707170 20903 webserver.cc:460] Webserver started at http://127.8.195.130:45585/ using document root <none> and password file <none>
I20260812 16:52:27.707917 20903 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:27.708076 20903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:27.708402 20903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:27.712050 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "f2af2a9e59a84f0a9e220ce3a91987c6"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "51ea9f7b86e801e2f4fc2793f5611c37"
server_key_iv: "a478b89feae281f351b6288d081a043d"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.712911 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "f2af2a9e59a84f0a9e220ce3a91987c6"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "51ea9f7b86e801e2f4fc2793f5611c37"
server_key_iv: "a478b89feae281f351b6288d081a043d"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.718338 20903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 16:52:27.721904 20935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.723351 20903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:27.723546 20903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "f2af2a9e59a84f0a9e220ce3a91987c6"
format_stamp: "Formatted at 2026-08-12 16:52:27 on dist-test-slave-z8x9"
server_key: "51ea9f7b86e801e2f4fc2793f5611c37"
server_key_iv: "a478b89feae281f351b6288d081a043d"
server_key_version: "encryptionkey@0"
I20260812 16:52:27.723783 20903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:27.748251 20903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:27.752997 20903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:27.753417 20903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:27.754562 20903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:27.756042 20903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:27.756119 20903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.756208 20903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:27.756248 20903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:27.790822 20903 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:46643
I20260812 16:52:27.790865 21195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:46643 every 8 connection(s)
I20260812 16:52:27.792129 20903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:27.801374 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20903
I20260812 16:52:27.801571 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:27.802019 8974 external_mini_cluster.cc:1428] Setting key 7bc0b551acc22bc8ded60db9df4b361d
Aug 12 16:52:27 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553547, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:27.814666 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:27.794760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:48911 (local address 127.8.195.190:37609)
0812 16:52:27.795240 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:27.795244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:27.796097 (+ 853us) server_negotiation.cc:407] Connection header received
0812 16:52:27.798007 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:27.798013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:27.798106 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:27.798229 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:27.799729 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.800469 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.802369 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.803038 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.803178 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:27.807170 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:27.807179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:27.807186 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:27.807236 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:27.811001 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:27.811803 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:27.811809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:27.811815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:27.811987 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:27.812459 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:27.812466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:27.812472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:27.812875 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:27.813033 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:27.813882 (+ 849us) server_negotiation.cc:299] Negotiation successful
0812 16:52:27.814147 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
I20260812 16:52:27.815487 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:27.795212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37609 (local address 127.8.195.130:48911)
0812 16:52:27.795931 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:27.795973 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:27.797740 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:27.798472 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:27.798480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:27.798991 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:27.799552 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:27.799569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.800638 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.800642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:27.802200 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:27.802213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:27.803286 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:27.803294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:27.803544 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:27.804290 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:27.804315 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:27.806892 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:27.811167 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:27.811176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:27.811191 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:27.811663 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:27.812107 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:27.812111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:27.812113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:27.812337 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:27.813026 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:27.813032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:27.813477 (+ 445us) client_negotiation.cc:769] Sending connection context
0812 16:52:27.813724 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 16:52:27.814142 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":286,"threads_started":1}
I20260812 16:52:27.817056 21199 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37609
I20260812 16:52:27.817433 21199 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:27.818208 21199 heartbeater.cc:511] Master 127.8.195.190:37609 requested a full tablet report, sending...
I20260812 16:52:27.819736 20479 ts_manager.cc:194] Registered new tserver with Master: f2af2a9e59a84f0a9e220ce3a91987c6 (127.8.195.130:46643)
I20260812 16:52:27.820585 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:48911
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:52:27.860994 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:28.204905 21209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:28.205260 21209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:28.205464 21209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:28.212901 21209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:28.213027 21209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:28.213269 21209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:28.223260 21209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:28.224618 21209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:28.226039 21209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:28.240289 21216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:28.240149 21218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:28.240491 21221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:28.241539 21209 server_base.cc:1034] running on GCE node
I20260812 16:52:28.242167 21209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:28.243157 21209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:28.244378 21209 hybrid_clock.cc:648] HybridClock initialized: now 1786553548244337 us; error 29 us; skew 500 ppm
I20260812 16:52:28.244676 21209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.248178 21209 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:52:28.249898 21209 webserver.cc:460] Webserver started at http://127.8.195.131:40161/ using document root <none> and password file <none>
I20260812 16:52:28.250633 21209 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:28.250770 21209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:28.251119 21209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:28.254369 21209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "c96ff1b50d2e438b8c50df4df1368000"
format_stamp: "Formatted at 2026-08-12 16:52:28 on dist-test-slave-z8x9"
server_key: "5fa33147721c492beb0b6294c77d9f32"
server_key_iv: "c4b0db44d3b356dfe5fefb71a0fa14ad"
server_key_version: "encryptionkey@0"
I20260812 16:52:28.255213 21209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "c96ff1b50d2e438b8c50df4df1368000"
format_stamp: "Formatted at 2026-08-12 16:52:28 on dist-test-slave-z8x9"
server_key: "5fa33147721c492beb0b6294c77d9f32"
server_key_iv: "c4b0db44d3b356dfe5fefb71a0fa14ad"
server_key_version: "encryptionkey@0"
I20260812 16:52:28.261053 21209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:52:28.264846 21231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:28.266727 21209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:28.266947 21209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "c96ff1b50d2e438b8c50df4df1368000"
format_stamp: "Formatted at 2026-08-12 16:52:28 on dist-test-slave-z8x9"
server_key: "5fa33147721c492beb0b6294c77d9f32"
server_key_iv: "c4b0db44d3b356dfe5fefb71a0fa14ad"
server_key_version: "encryptionkey@0"
I20260812 16:52:28.267189 21209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:28.279415 21209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:28.284153 21209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:28.284482 21209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:28.285547 21209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:28.286969 21209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:28.287072 21209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:28.287185 21209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:28.287240 21209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:28.288888 20887 heartbeater.cc:503] Master 127.8.195.190:37609 was elected leader, sending a full tablet report...
I20260812 16:52:28.323768 21209 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:45797
I20260812 16:52:28.323863 21467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:45797 every 8 connection(s)
I20260812 16:52:28.325093 21209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:28.328260 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21209
I20260812 16:52:28.328459 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:28.329087 8974 external_mini_cluster.cc:1428] Setting key 75891b6d58366301c12148beed57b518
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:28.346642 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.327802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:45541 (local address 127.8.195.190:37609)
0812 16:52:28.328304 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.328309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.329528 (+ 1219us) server_negotiation.cc:407] Connection header received
0812 16:52:28.331686 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.331692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.331790 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.331914 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.333628 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.334430 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.336000 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.336408 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.336546 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.340353 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.340358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.340361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.340396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.343343 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.344242 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.344247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.344250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.344430 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.344988 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.344992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.344995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.345367 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.345508 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.346190 (+ 682us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.346374 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":254,"threads_started":1}
I20260812 16:52:28.347595 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.328474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37609 (local address 127.8.195.131:45541)
0812 16:52:28.329365 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:28.329405 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:28.331418 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:28.332179 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:28.332193 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:28.332799 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:28.333441 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:28.333461 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.334592 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.334596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:28.335852 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:28.335864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.336653 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.336661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:28.336894 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.337664 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:28.337691 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:28.340086 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:28.343484 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:28.343494 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:28.343511 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:28.344103 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:28.344548 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:28.344552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:28.344556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:28.344832 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:28.345493 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:28.345500 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:28.345836 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 16:52:28.346078 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 16:52:28.346495 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":331,"threads_started":1}
I20260812 16:52:28.349212 21472 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37609
I20260812 16:52:28.349524 21472 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:28.350229 21472 heartbeater.cc:511] Master 127.8.195.190:37609 requested a full tablet report, sending...
I20260812 16:52:28.351642 20479 ts_manager.cc:194] Registered new tserver with Master: c96ff1b50d2e438b8c50df4df1368000 (127.8.195.131:45797)
I20260812 16:52:28.352442 20479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:45541
I20260812 16:52:28.357648 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/test-admin.keytab.
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.391911 8974 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 16:52:28.397305 8974 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc' of type FILE
W20260812 16:52:28.397358 21483 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:28.394671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.8.195.129:46445)
0812 16:52:28.395371 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.395380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.395411 (+ 31us) server_negotiation.cc:407] Connection header received
0812 16:52:28.395474 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.395481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.395694 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.395833 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.396800 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49002: BlockingRecv error: recv got EOF from 127.0.0.1:49002 (error 108)
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":386,"threads_started":1}
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.399001 8974 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.400527 8974 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:28.416914 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.402475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.8.195.129:46445)
0812 16:52:28.402704 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.402713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.402986 (+ 273us) server_negotiation.cc:407] Connection header received
0812 16:52:28.403197 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.403201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.403265 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.403389 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.404457 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.405524 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.406795 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.407378 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.407516 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.411317 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.411348 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.411364 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.411399 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.414275 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.415035 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.415043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.415045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.415173 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.415529 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.415532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.415534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.415898 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.416110 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.416425 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.416635 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 16:52:28.423461 21483 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:28.421510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49016 (local address 127.8.195.129:46445)
0812 16:52:28.421745 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.421777 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.422014 (+ 237us) server_negotiation.cc:407] Connection header received
0812 16:52:28.422204 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.422209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.422291 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.422395 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.423216 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49016: BlockingRecv error: recv got EOF from 127.0.0.1:49016 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260812 16:52:28.423652 8974 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.425372 8974 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.426755 8974 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:28.441900 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.428172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.8.195.129:46445)
0812 16:52:28.428373 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.428379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.428634 (+ 255us) server_negotiation.cc:407] Connection header received
0812 16:52:28.428763 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.428767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.428822 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.428902 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.429930 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.430626 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.432017 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.432468 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.432575 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.436302 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.436307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.436311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.436362 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.439475 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.440178 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.440183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.440185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.440341 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.440705 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.440709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.440712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.440975 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.441284 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.441509 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.441673 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 16:52:28.446760 21483 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:28.444816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.8.195.129:46445)
0812 16:52:28.445042 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.445048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.445348 (+ 300us) server_negotiation.cc:407] Connection header received
0812 16:52:28.445551 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.445556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.445627 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.445736 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.446525 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49040: BlockingRecv error: recv got EOF from 127.0.0.1:49040 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 16:52:28.446991 8974 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786553491558846-8974-0/krb5kdc/krb5cc' of type FILE
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.448626 8974 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:28.450166 8974 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:28.465595 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.451544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.8.195.129:46445)
0812 16:52:28.451776 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.451783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.452074 (+ 291us) server_negotiation.cc:407] Connection header received
0812 16:52:28.452270 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.452275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.452350 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.452454 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.453585 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.454445 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.455668 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.456128 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.456281 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.460153 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.460158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.460162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.460197 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.462842 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.463492 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.463500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.463503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.463650 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.464040 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.464045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.464048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.464363 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.464703 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.465020 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.465289 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:52:28.467446 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20594
I20260812 16:52:28.479838 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.468203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.8.195.129:46445)
0812 16:52:28.468362 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.468366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.468832 (+ 466us) server_negotiation.cc:407] Connection header received
0812 16:52:28.468952 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.468956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.469012 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.469089 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.470051 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.470895 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.472352 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.472737 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.472846 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.474978 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.474983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.474986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.475018 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.477605 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.478213 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.478216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.478218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.478374 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.478683 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.478686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.478688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.478935 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.479140 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.479423 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.479587 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:52:28.480437 20877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:28.564952 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20594
I20260812 16:52:28.593211 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20903
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:52:28.611104 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.594178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.8.195.130:46643)
0812 16:52:28.594748 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.594755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.594782 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:52:28.594844 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.594852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.595130 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.595260 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.596632 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.597880 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.599687 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.600238 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.600362 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.604870 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.604882 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.604897 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.604938 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.608014 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.608732 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.608740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.608743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.608952 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.609441 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.609446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.609449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.609844 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.610173 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.610508 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.610778 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":305,"threads_started":1}
I20260812 16:52:28.612409 21185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:28.701332 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20903
I20260812 16:52:28.732059 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21209
Aug 12 16:52:28 dist-test-slave-z8x9 krb5kdc[20395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:52:28.749064 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:28.733487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.8.195.131:45797)
0812 16:52:28.734260 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:28.734268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:28.734300 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:52:28.734370 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:28.734378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:28.734616 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:28.734766 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:28.735867 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.737041 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.738523 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:28.739129 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:28.739297 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:28.743440 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:28.743452 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:28.743467 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:28.743504 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:28.746487 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.747136 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.747144 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.747146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.747270 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:28.747672 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:28.747676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:28.747678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:28.748052 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:28.748313 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:28.748555 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:52:28.748780 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":393,"threads_started":1}
I20260812 16:52:28.750398 21459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:28.837255 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21209
I20260812 16:52:28.866212 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20416
I20260812 16:52:28.867393 20533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:28.972265 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20416
2026-08-12T16:52:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4520 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:29 dist-test-slave-z8x9 krb5kdc[21523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:29 dist-test-slave-z8x9 krb5kdc[21523](info): set up 2 sockets
Aug 12 16:52:29 dist-test-slave-z8x9 krb5kdc[21523](info): commencing operation
krb5kdc: starting...
W20260812 16:52:30.374341 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:30 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab.
Entry for principal oryx/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab.
I20260812 16:52:30.636426 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35251
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab
--principal=oryx/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:30.981000 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:30.981458 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:30.981637 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:30.989089 21549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:30.989312 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:30.989421 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:30.989522 21549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:30.989610 21549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:30.999802 21549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab
--principal=oryx/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:31.001148 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:31.002547 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:31.014748 21559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:31.014858 21560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:31.014782 21563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:31.015623 21549 server_base.cc:1034] running on GCE node
I20260812 16:52:31.016482 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:31.018045 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:31.019304 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786553551019269 us; error 63 us; skew 500 ppm
I20260812 16:52:31.019676 21549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:31 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:31.023082 21549 init.cc:377] Logged in from keytab as oryx/127.8.195.190@KRBTEST.COM (short username oryx)
I20260812 16:52:31.024736 21549 webserver.cc:460] Webserver started at http://127.8.195.190:43005/ using document root <none> and password file <none>
I20260812 16:52:31.025527 21549 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:31.025684 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:31.026005 21549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:31.029014 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "850b1174ee3e4e81b8e79029d9804668"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "532282cd916afd84e0e17e061934ced5"
server_key_iv: "18035292bf76270d1980fe078b16498e"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.029923 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "850b1174ee3e4e81b8e79029d9804668"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "532282cd916afd84e0e17e061934ced5"
server_key_iv: "18035292bf76270d1980fe078b16498e"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.035699 21549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:31.039227 21581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:31.040575 21549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:31.040776 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "850b1174ee3e4e81b8e79029d9804668"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "532282cd916afd84e0e17e061934ced5"
server_key_iv: "18035292bf76270d1980fe078b16498e"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.040982 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:31.095211 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:31.099859 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:31.100179 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:31.120448 21549 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:35251
I20260812 16:52:31.120472 21662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:35251 every 8 connection(s)
I20260812 16:52:31.121953 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:31.123555 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21549
I20260812 16:52:31.123796 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:31.124677 8974 external_mini_cluster.cc:1428] Setting key 7908a8e7bb40d7aecacb542c331ee4ff
I20260812 16:52:31.127015 21663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:52:31 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:31.140909 21663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Bootstrap starting.
I20260812 16:52:31.144516 21663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:31.145756 21663 log.cc:824] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:31.149384 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:31.126916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.8.195.190:35251)
0812 16:52:31.127769 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:31.127780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:31.127820 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:52:31.129496 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:31.129517 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:31.129895 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:31.130219 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:31.132167 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.133611 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.135420 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.136077 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.136209 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:31.140993 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:31.141001 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:31.141014 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:31.141076 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:31.144224 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:31.145091 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:31.145098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:31.145103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:31.145344 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:31.145816 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:31.145822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:31.145825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:31.146599 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:31.146845 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:31.147486 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 16:52:31.147943 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":415,"threads_started":1}
I20260812 16:52:31.149423 21663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: No bootstrap required, opened a new log
I20260812 16:52:31.160727 21663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } }
I20260812 16:52:31.161245 21663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:31.161370 21663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 850b1174ee3e4e81b8e79029d9804668, State: Initialized, Role: FOLLOWER
I20260812 16:52:31.161900 21663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } }
I20260812 16:52:31.162115 21663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:31.162222 21663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:31.162351 21663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:31.164774 21663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } }
I20260812 16:52:31.165321 21663 leader_election.cc:302] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 850b1174ee3e4e81b8e79029d9804668; no voters:
I20260812 16:52:31.165886 21663 leader_election.cc:288] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:31.166055 21669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:31.166774 21669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 LEADER]: Becoming Leader. State: Replica: 850b1174ee3e4e81b8e79029d9804668, State: Running, Role: LEADER
I20260812 16:52:31.167336 21669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } }
I20260812 16:52:31.168061 21663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:31.171123 21671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850b1174ee3e4e81b8e79029d9804668. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } }
I20260812 16:52:31.171094 21670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } }
I20260812 16:52:31.171528 21671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:31.171655 21670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:31.172616 21682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:31.182210 21682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:31.193313 21682 catalog_manager.cc:1347] Generated new cluster ID: ed653dec459d419f95caf2e8ef0d233a
I20260812 16:52:31.193466 21682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:31.214368 21682 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:31.216092 21682 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:31.225256 21682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Generated new TSK 0
I20260812 16:52:31.226503 21682 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
I20260812 16:52:31.305227 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:31.659909 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:31.660275 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:31.660498 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:31.668503 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:31.668629 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:31.668809 21711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:31.679345 21711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:31.680742 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:31.682318 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:31.696192 21724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:31.696252 21725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:31.696134 21728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:31.697237 21711 server_base.cc:1034] running on GCE node
I20260812 16:52:31.697932 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:31.698966 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:31.700218 21711 hybrid_clock.cc:648] HybridClock initialized: now 1786553551700148 us; error 28 us; skew 500 ppm
I20260812 16:52:31.700598 21711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:31 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:31.707787 21711 init.cc:377] Logged in from keytab as oryx/127.8.195.129@KRBTEST.COM (short username oryx)
I20260812 16:52:31.709540 21711 webserver.cc:460] Webserver started at http://127.8.195.129:36305/ using document root <none> and password file <none>
I20260812 16:52:31.710340 21711 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:31.710481 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:31.710824 21711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:31.714098 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "941321848a1b4dcf9a124db4ad04fb1e"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "59827c98bfb0e1b163c3d721bea50337"
server_key_iv: "bf293eb052d787ec2e2218e2d0f915fe"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.714901 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "941321848a1b4dcf9a124db4ad04fb1e"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "59827c98bfb0e1b163c3d721bea50337"
server_key_iv: "bf293eb052d787ec2e2218e2d0f915fe"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.720840 21711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:52:31.724852 21739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:31.726418 21711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:31.726603 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "941321848a1b4dcf9a124db4ad04fb1e"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "59827c98bfb0e1b163c3d721bea50337"
server_key_iv: "bf293eb052d787ec2e2218e2d0f915fe"
server_key_version: "encryptionkey@0"
I20260812 16:52:31.726807 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:31.766021 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:31.770931 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:31.771281 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:31.772368 21711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:31.773973 21711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:31.774061 21711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:31.774187 21711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:31.774251 21711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:31.809098 21711 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:45367
I20260812 16:52:31.809146 21973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:45367 every 8 connection(s)
I20260812 16:52:31.810463 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:31.814525 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21711
I20260812 16:52:31.814776 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:31.815644 8974 external_mini_cluster.cc:1428] Setting key 73a856b2959acb9b49e9fd0b948f291d
Aug 12 16:52:31 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:31.835285 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:31.813437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:58225 (local address 127.8.195.190:35251)
0812 16:52:31.813903 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:31.813908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:31.815070 (+ 1162us) server_negotiation.cc:407] Connection header received
0812 16:52:31.817550 (+ 2480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:31.817554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:31.817652 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:31.817760 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:31.819301 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.820498 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.822263 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.822983 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.823139 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:31.827752 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:31.827762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:31.827769 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:31.827818 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:31.830959 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:31.832211 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:31.832217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:31.832219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:31.832387 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:31.833024 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:31.833028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:31.833031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:31.833504 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:31.833637 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:31.834793 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0812 16:52:31.835014 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":240,"threads_started":1}
I20260812 16:52:31.837003 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:31.813977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:58225)
0812 16:52:31.814884 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:31.814934 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:31.817257 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:31.818036 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:31.818047 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:31.818566 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:31.819118 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:31.819135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.820680 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.820693 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:31.822060 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:31.822072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:31.823273 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:31.823280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:31.823522 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:31.824307 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:31.824332 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:31.827423 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:31.831150 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:31.831163 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:31.831186 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:31.832034 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:31.832531 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:31.832539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:31.832545 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:31.832870 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:31.833675 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:31.833687 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:31.834226 (+ 539us) client_negotiation.cc:769] Sending connection context
0812 16:52:31.834642 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 16:52:31.835316 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":361,"threads_started":1}
I20260812 16:52:31.838893 21976 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:31.839265 21976 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:31.840087 21976 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:31.842878 21609 ts_manager.cc:194] Registered new tserver with Master: 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:31.845445 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:58225
WARNING: no policy specified for oryx/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
I20260812 16:52:31.881426 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:32.234691 21990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:32.234999 21990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:32.235159 21990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:32.242616 21990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:32.242743 21990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:32.242937 21990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:32.252970 21990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:32.254356 21990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:32.255710 21990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:32.269637 22002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:32.269685 21998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:32.269724 22008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:32.270592 21990 server_base.cc:1034] running on GCE node
I20260812 16:52:32.271257 21990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:32.272213 21990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:32.273458 21990 hybrid_clock.cc:648] HybridClock initialized: now 1786553552273389 us; error 60 us; skew 500 ppm
I20260812 16:52:32.273849 21990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:32 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:32.277383 21990 init.cc:377] Logged in from keytab as oryx/127.8.195.130@KRBTEST.COM (short username oryx)
I20260812 16:52:32.278934 21990 webserver.cc:460] Webserver started at http://127.8.195.130:42173/ using document root <none> and password file <none>
I20260812 16:52:32.279665 21990 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:32.279832 21990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:32.280158 21990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:32.283272 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "043a4c55c2e4a2f36e578a551670cd71"
server_key_iv: "4334c566ca994ff5ab2b52d3903e7d65"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.284125 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "043a4c55c2e4a2f36e578a551670cd71"
server_key_iv: "4334c566ca994ff5ab2b52d3903e7d65"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.289958 21990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:52:32.293823 22019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.295239 21990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:32.295421 21990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "043a4c55c2e4a2f36e578a551670cd71"
server_key_iv: "4334c566ca994ff5ab2b52d3903e7d65"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.295635 21990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:32.309653 21990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:32.314738 21990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:32.315092 21990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:32.316107 21990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:32.317656 21990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:32.317785 21990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.317930 21990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:32.318007 21990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.352887 21990 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40881
I20260812 16:52:32.352934 22244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40881 every 8 connection(s)
I20260812 16:52:32.354266 21990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:32.357698 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21990
I20260812 16:52:32.357898 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:32.358532 8974 external_mini_cluster.cc:1428] Setting key 2e10667fe8ce88d9447da07f3c5ae75b
Aug 12 16:52:32 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:32.379977 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:32.356980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:35333 (local address 127.8.195.190:35251)
0812 16:52:32.357594 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:32.357602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:32.358649 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 16:52:32.360592 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:32.360597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:32.360681 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:32.360780 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:32.363016 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.363761 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.366427 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.366884 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.367003 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:32.372022 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:32.372031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:32.372039 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:32.372088 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:32.376206 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:32.377068 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:32.377077 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:32.377084 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:32.377312 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:32.377812 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:32.377820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:32.377826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:32.378241 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:32.378424 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:32.379155 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 16:52:32.379424 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":338,"threads_started":1}
I20260812 16:52:32.380731 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:32.357547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:35333)
0812 16:52:32.358438 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:32.358489 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:32.360325 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:32.361081 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:32.361092 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:32.361937 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:32.362816 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:32.362847 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.363927 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.363935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:32.366230 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:32.366250 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.367132 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.367142 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:32.367508 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:32.368350 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:32.368387 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:32.371663 (+ 3276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:32.376402 (+ 4739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:32.376411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:32.376426 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:32.376909 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:32.377454 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:32.377458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:32.377460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:32.377668 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:32.378394 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:32.378401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:32.378766 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 16:52:32.379001 (+ 235us) client_negotiation.cc:240] Negotiation successful
0812 16:52:32.379435 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":366,"threads_started":1}
I20260812 16:52:32.382493 22246 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:32.382833 22246 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:32.383623 22246 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:32.385473 21609 ts_manager.cc:194] Registered new tserver with Master: f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:32.386328 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:35333
WARNING: no policy specified for oryx/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
I20260812 16:52:32.424973 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:32.777462 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:32.777789 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:32.777961 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:32.785341 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:32.785498 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:32.785691 22253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:32.796089 22253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:32.797559 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:32.799001 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:32.814692 22263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:32.814625 22264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:32.815160 22268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:32.817073 22253 server_base.cc:1034] running on GCE node
I20260812 16:52:32.817800 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:32.818985 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:32.820268 22253 hybrid_clock.cc:648] HybridClock initialized: now 1786553552820196 us; error 64 us; skew 500 ppm
I20260812 16:52:32.820658 22253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:32 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:32.824368 22253 init.cc:377] Logged in from keytab as oryx/127.8.195.131@KRBTEST.COM (short username oryx)
I20260812 16:52:32.826259 22253 webserver.cc:460] Webserver started at http://127.8.195.131:36869/ using document root <none> and password file <none>
I20260812 16:52:32.827103 22253 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:32.827270 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:32.827615 22253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:32.830951 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "b1030e057604403ab2db3cb7898edb1c"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "3a8dc5cc211f529b4e16ea01a5ae87c2"
server_key_iv: "ea50f8a2735e14a22bd2909d2450c72a"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.831898 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "b1030e057604403ab2db3cb7898edb1c"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "3a8dc5cc211f529b4e16ea01a5ae87c2"
server_key_iv: "ea50f8a2735e14a22bd2909d2450c72a"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.837989 22253 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 16:52:32.841753 22285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.843176 22253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:52:32.843364 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "b1030e057604403ab2db3cb7898edb1c"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "3a8dc5cc211f529b4e16ea01a5ae87c2"
server_key_iv: "ea50f8a2735e14a22bd2909d2450c72a"
server_key_version: "encryptionkey@0"
I20260812 16:52:32.843598 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:32.849488 21976 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
I20260812 16:52:32.865417 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:32.871548 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:32.871917 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:32.873291 22253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:32.875380 22253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:32.875488 22253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.875623 22253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:32.875692 22253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:32.914397 22253 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:35483
I20260812 16:52:32.914409 22483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:35483 every 8 connection(s)
I20260812 16:52:32.916296 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:32.923517 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 22253
I20260812 16:52:32.923738 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:32.924391 8974 external_mini_cluster.cc:1428] Setting key 10a7efe60b3578b1643cc02b8f84ade8
Aug 12 16:52:32 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:32.941978 22491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:32.920724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:35315 (local address 127.8.195.190:35251)
0812 16:52:32.921256 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:32.921264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:32.921371 (+ 107us) server_negotiation.cc:407] Connection header received
0812 16:52:32.923388 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:32.923393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:32.923505 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:32.923629 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:32.925588 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.926606 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.928579 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.929378 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.929538 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:32.934324 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:32.934330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:32.934334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:32.934384 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:32.937510 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:32.938752 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:32.938759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:32.938762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:32.938945 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:32.939586 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:32.939593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:32.939597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:32.940083 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:32.940284 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:32.941307 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0812 16:52:32.941566 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":236,"threads_started":1}
I20260812 16:52:32.943113 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:32.920077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:35315)
0812 16:52:32.921135 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:32.921225 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:32.923082 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:32.923990 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:32.924000 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:32.924621 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:32.925387 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:32.925424 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.926781 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.926794 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:32.928402 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:32.928419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:32.929635 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:32.929641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:32.929872 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:32.930733 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:32.930762 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:32.933860 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:32.937687 (+ 3827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:32.937702 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:32.937727 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:32.938543 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:32.939083 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:32.939095 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:32.939098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:32.939416 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:32.940233 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:32.940244 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:32.940769 (+ 525us) client_negotiation.cc:769] Sending connection context
0812 16:52:32.941315 (+ 546us) client_negotiation.cc:240] Negotiation successful
0812 16:52:32.941919 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":410,"threads_started":1}
I20260812 16:52:32.944784 22486 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:32.945197 22486 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:32.946213 22486 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:32.948096 21609 ts_manager.cc:194] Registered new tserver with Master: b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483)
I20260812 16:52:32.949566 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:35315
I20260812 16:52:32.957010 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.195.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.189.keytab.
Entry for principal oryx/127.8.195.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.189.keytab.
I20260812 16:52:33.025353 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:46093
--webserver_interface=127.8.195.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.189.keytab
--principal=oryx/127.8.195.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35251,127.8.195.189:46093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:33.387892 22495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:33.388208 22495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:33.388373 22495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 16:52:33.389094 22246 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
W20260812 16:52:33.396421 22495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:33.396592 22495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:33.396706 22495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:33.396812 22495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:33.396895 22495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:33.407405 22495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35251,127.8.195.189:46093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.189.keytab
--principal=oryx/127.8.195.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:46093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.195.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
W20260812 16:52:33.407851 22495 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.195.190:35251,127.8.195.189:46093'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 16:52:33.408998 22495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:33.410425 22495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:33.423048 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:33.423173 22512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:33.423028 22510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:33.424355 22495 server_base.cc:1034] running on GCE node
I20260812 16:52:33.425402 22495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:33.427042 22495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:33.428315 22495 hybrid_clock.cc:648] HybridClock initialized: now 1786553553428245 us; error 53 us; skew 500 ppm
I20260812 16:52:33.428718 22495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553553, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:33.433617 22495 init.cc:377] Logged in from keytab as oryx/127.8.195.189@KRBTEST.COM (short username oryx)
I20260812 16:52:33.443630 22495 webserver.cc:460] Webserver started at http://127.8.195.189:37543/ using document root <none> and password file <none>
I20260812 16:52:33.444941 22495 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:33.445116 22495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:33.445695 22495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:33.451323 22495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/instance:
uuid: "835cac8d50ab4eb59fe5537783732606"
format_stamp: "Formatted at 2026-08-12 16:52:33 on dist-test-slave-z8x9"
server_key: "f155f5706254bf96780688b79bb6f0b5"
server_key_iv: "10de1b7e732a80c7d9ab7e7779be9184"
server_key_version: "encryptionkey@0"
I20260812 16:52:33.452629 22495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal/instance:
uuid: "835cac8d50ab4eb59fe5537783732606"
format_stamp: "Formatted at 2026-08-12 16:52:33 on dist-test-slave-z8x9"
server_key: "f155f5706254bf96780688b79bb6f0b5"
server_key_iv: "10de1b7e732a80c7d9ab7e7779be9184"
server_key_version: "encryptionkey@0"
I20260812 16:52:33.461119 22495 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260812 16:52:33.466257 22532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:33.468322 22495 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 16:52:33.468647 22495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "835cac8d50ab4eb59fe5537783732606"
format_stamp: "Formatted at 2026-08-12 16:52:33 on dist-test-slave-z8x9"
server_key: "f155f5706254bf96780688b79bb6f0b5"
server_key_iv: "10de1b7e732a80c7d9ab7e7779be9184"
server_key_version: "encryptionkey@0"
I20260812 16:52:33.468987 22495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:33.518934 22495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:33.525887 22495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:33.526289 22495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:33.548923 22495 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.189:46093
I20260812 16:52:33.548926 22635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.189:46093 every 8 connection(s)
I20260812 16:52:33.550493 22495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
I20260812 16:52:33.555855 22636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:33.556063 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 22495
I20260812 16:52:33.568831 22636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553553, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.189@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:33.584477 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.558486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.8.195.189:46093)
0812 16:52:33.559402 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.559414 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.559461 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:52:33.561824 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.561864 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.562336 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.562752 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.564553 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.565997 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.567345 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.568145 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.568345 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.573515 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.573530 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.573557 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.573627 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.578517 (+ 4890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.579464 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.579473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.579482 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.579714 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.580240 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.580245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.580249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.581089 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.581584 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.582292 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.582809 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":408,"threads_started":1}
I20260812 16:52:33.591786 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.572744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.8.195.190:35251)
0812 16:52:33.573402 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.573407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.575563 (+ 2156us) server_negotiation.cc:407] Connection header received
0812 16:52:33.575635 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.575642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.575788 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.575920 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.577620 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.578420 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.580311 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.580730 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.580877 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.584990 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.584999 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.585003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.585043 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.588726 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.589767 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.589771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.589774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.589909 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.590325 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.590328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.590330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.590676 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.590797 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.591324 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.591540 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":287,"threads_started":1}
I20260812 16:52:33.592020 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.572749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34576)
0812 16:52:33.575183 (+ 2434us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.575240 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.575424 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.576150 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.576245 (+ 95us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.577043 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.577450 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.577461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.578572 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.578577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.580137 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.580154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.580937 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.580944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.581157 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.582093 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.582119 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.584492 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.588878 (+ 4386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.588890 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.588897 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.589597 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.590027 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.590033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.590037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.590208 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.590836 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.590843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.591011 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.591200 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.591600 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":1555,"threads_started":1}
I20260812 16:52:33.594058 22636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553553, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.189@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
I20260812 16:52:33.608666 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.594949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57106)
0812 16:52:33.595196 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.595217 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.595371 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.595680 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.595686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.596234 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.596451 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.596464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.597556 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.597561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.598987 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.599000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.599720 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.599726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.599954 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.600610 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.600632 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.603148 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.606136 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.606143 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.606147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.606555 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.606931 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.606938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.606941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.607126 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.607664 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.607669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.607808 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.607960 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.608267 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 16:52:33.608888 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.595171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57106 (local address 127.8.195.189:46093)
0812 16:52:33.595322 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.595326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.595343 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:52:33.595485 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.595489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.595547 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.595624 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.596599 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.597411 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.599114 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.599561 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.599658 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.603334 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.603340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.603345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.603385 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.605994 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.606682 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.606689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.606697 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.606818 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.607240 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.607247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.607250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.607547 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.607701 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.608037 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.608422 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":54}
I20260812 16:52:33.613847 22636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Bootstrap starting.
I20260812 16:52:33.618090 22636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:33.619717 22636 log.cc:824] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:33.624197 22636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: No bootstrap required, opened a new log
I20260812 16:52:33.634826 22636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } }
I20260812 16:52:33.635301 22636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:33.635443 22636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835cac8d50ab4eb59fe5537783732606, State: Initialized, Role: FOLLOWER
I20260812 16:52:33.636325 22636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } }
I20260812 16:52:33.636670 22649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:33.637141 22649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } }
I20260812 16:52:33.638144 22652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } } }
I20260812 16:52:33.638664 22652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:33.639197 22649 leader_election.cc:288] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251)
I20260812 16:52:33.639578 22636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:33.640203 21620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "835cac8d50ab4eb59fe5537783732606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "850b1174ee3e4e81b8e79029d9804668" is_pre_election: true
I20260812 16:52:33.641805 22533 leader_election.cc:302] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 835cac8d50ab4eb59fe5537783732606; no voters: 850b1174ee3e4e81b8e79029d9804668
I20260812 16:52:33.642782 22649 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 16:52:33.650635 22667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:52:33.650858 22667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:52:33.654175 22495 master_runner.cc:428] Detected that this master 835cac8d50ab4eb59fe5537783732606 is joining an existing cluster
I20260812 16:52:33.654361 22495 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.195.189:46093
I20260812 16:52:33.679780 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.659168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.8.195.190:35251)
0812 16:52:33.659414 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.659430 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.659964 (+ 534us) server_negotiation.cc:407] Connection header received
0812 16:52:33.660313 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.660319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.660420 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.660506 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.662631 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.664171 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.667039 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.667773 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.667882 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.671029 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.671036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.671041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.671082 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.675021 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.676400 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.676406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.676410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.676586 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.677092 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.677098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.677102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.677587 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.677731 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.678785 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.679062 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 16:52:33.681008 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.659150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34580)
0812 16:52:33.659839 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.659860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.660101 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.660769 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.660777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.661839 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.662390 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.662409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.664473 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.664481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.666813 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.666836 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.668239 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.668255 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.668447 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.669322 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.669345 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.670788 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.675256 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.675263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.675266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.676223 (+ 957us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.676753 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.676757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.676760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.676951 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.677815 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.677821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.678064 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.680094 (+ 2030us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.680471 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":372,"threads_started":1}
I20260812 16:52:33.686053 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.660122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.8.195.189:46093)
0812 16:52:33.660371 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.660382 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.667579 (+ 7197us) server_negotiation.cc:407] Connection header received
0812 16:52:33.668052 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.668058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.668139 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.668215 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.670042 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.671259 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.673546 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.674300 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.674414 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.677431 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.677438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.677443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.677478 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.682103 (+ 4625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.682972 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.682977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.682980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.683170 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.683699 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.683705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.683710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.684054 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.684237 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.685142 (+ 905us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.685641 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:52:33.687143 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.660545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57110)
0812 16:52:33.667465 (+ 6920us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.667492 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.667632 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.668544 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.668550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.669451 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.669884 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.669898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.671474 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.671479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.673358 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.673378 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.674798 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.674806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.675000 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.675752 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.675777 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.677248 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.682249 (+ 5001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.682256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.682260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.682830 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.683398 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.683403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.683410 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.683583 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.684307 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.684311 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.684493 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.686326 (+ 1833us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.686678 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6761,"thread_start_us":7190,"threads_started":1}
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.190@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
I20260812 16:52:33.709954 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.689700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57120 (local address 127.8.195.189:46093)
0812 16:52:33.689889 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.689895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.690111 (+ 216us) server_negotiation.cc:407] Connection header received
0812 16:52:33.690596 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.690601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.690732 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.690813 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.692728 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.694065 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.696378 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.697312 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.697432 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.702575 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.702585 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.702592 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.702639 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.706340 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.707118 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.707125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.707129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.707286 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.707850 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.707858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.707864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.708289 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.708420 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.709151 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.709483 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 16:52:33.711293 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.689373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57120)
0812 16:52:33.689976 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.690027 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.690294 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.691115 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.691123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.692142 (+ 1019us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.692514 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.692530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.694314 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.694322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.696188 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.696210 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.697773 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.697781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.698036 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.698920 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.698951 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.702323 (+ 3372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.706495 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.706520 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.706526 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.706980 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.707471 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.707479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.707485 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.707708 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.708536 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.708547 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.708790 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.710293 (+ 1503us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.710732 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":11926,"threads_started":1}
I20260812 16:52:33.713824 21609 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.195.189:46093: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "835cac8d50ab4eb59fe5537783732606"
member_type: NON_VOTER
last_known_addr {
host: "127.8.195.189"
port: 46093
}
attrs {
promote: true
}
}
dest_uuid: "850b1174ee3e4e81b8e79029d9804668"
cas_config_opid_index: -1
I20260812 16:52:33.715572 21609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.8.195.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.188@KRBTEST.COM" created.
I20260812 16:52:33.731223 22580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:33.730842 22687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 835cac8d50ab4eb59fe5537783732606 (127.8.195.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } }
I20260812 16:52:33.735548 22580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 16:52:33.735658 22687 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.195.189:46093
I20260812 16:52:33.739140 22495 master.cc:537] Master@127.8.195.189:46093 shutting down...
I20260812 16:52:33.743386 22686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:52:33.749997 21586 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 16:52:33.757679 22495 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:33.758193 22495 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:33.758410 22495 tablet_replica.cc:330] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: stopping tablet replica
I20260812 16:52:33.760942 22688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } }
I20260812 16:52:33.761420 22688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:33.772446 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.771700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57134)
0812 16:52:33.771930 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.772196 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:46093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 16:52:33.772761 21586 consensus_peers.cc:487] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 -> Peer 835cac8d50ab4eb59fe5537783732606 (127.8.195.189:46093): Couldn't send request to peer 835cac8d50ab4eb59fe5537783732606. Status: Network error: Client connection negotiation failed: client connection to 127.8.195.189:46093: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:52:33.774904 22495 master.cc:552] Master@127.8.195.189:46093 shutdown complete.
I20260812 16:52:33.775157 22495 master_runner.cc:305] Clearing existing system tablet
I20260812 16:52:33.775797 22495 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:33.775977 22495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:33.779433 22495 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.188.keytab.
Entry for principal oryx/127.8.195.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.188.keytab.
I20260812 16:52:33.781454 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:33.782624 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:38463
--webserver_interface=127.8.195.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.188.keytab
--principal=oryx/127.8.195.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:52:33.785616 22495 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:33.785960 22495 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260812 16:52:33.786067 22495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "835cac8d50ab4eb59fe5537783732606"
format_stamp: "Formatted at 2026-08-12 16:52:33 on dist-test-slave-z8x9"
server_key: "f155f5706254bf96780688b79bb6f0b5"
server_key_iv: "10de1b7e732a80c7d9ab7e7779be9184"
server_key_version: "encryptionkey@0"
I20260812 16:52:33.791831 22495 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:33.799320 22495 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:52:33.799533 22495 log.cc:1196] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 16:52:33.800027 22495 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Deleting consensus metadata
I20260812 16:52:33.800325 22495 master_runner.cc:315] Copying system tablet from 127.8.195.190:35251
I20260812 16:52:33.804036 22495 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: tablet copy: Beginning tablet copy session from remote peer at address 127.8.195.190:35251
I20260812 16:52:33.823755 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.805261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.8.195.190:35251)
0812 16:52:33.805502 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:33.805511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:33.805929 (+ 418us) server_negotiation.cc:407] Connection header received
0812 16:52:33.806232 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:33.806238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:33.806321 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:33.806395 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:33.807688 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.809035 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.811781 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.812654 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.812783 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.816258 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:33.816267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:33.816273 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:33.816320 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:33.820370 (+ 4050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.821306 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.821312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.821316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.821487 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:33.822010 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:33.822016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:33.822019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:33.822429 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:33.822593 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:33.823107 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 16:52:33.823360 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:52:33.824927 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:33.805201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34592)
0812 16:52:33.805812 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:33.805834 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:33.806052 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:33.806628 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:33.806633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:33.807209 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:33.807499 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.807514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.809767 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.809776 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.811552 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:33.811573 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:33.813135 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:33.813143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:33.813738 (+ 595us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:33.814548 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:33.814583 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:33.816000 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:33.820560 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.820566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.820571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.821134 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.821646 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:33.821652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:33.821655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:33.821854 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:33.822572 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:33.822577 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:33.822787 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 16:52:33.824058 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0812 16:52:33.824444 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":312,"threads_started":1}
I20260812 16:52:33.826066 21636 tablet_copy_service.cc:140] P 850b1174ee3e4e81b8e79029d9804668: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 835cac8d50ab4eb59fe5537783732606 ({username='oryx', principal='oryx/127.8.195.189@KRBTEST.COM'} at 127.0.0.1:34592)
I20260812 16:52:33.826489 21636 tablet_copy_service.cc:161] P 850b1174ee3e4e81b8e79029d9804668: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 835cac8d50ab4eb59fe5537783732606 at {username='oryx', principal='oryx/127.8.195.189@KRBTEST.COM'} at 127.0.0.1:34592: session id = 835cac8d50ab4eb59fe5537783732606-00000000000000000000000000000000
I20260812 16:52:33.832109 21636 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:52:33.835421 22495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:52:33.842865 22495 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: tablet copy: Starting download of 0 data blocks...
I20260812 16:52:33.843292 22495 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: tablet copy: Starting download of 1 WAL segments...
I20260812 16:52:33.847368 22495 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:52:33.852572 21636 tablet_copy_service.cc:342] P 850b1174ee3e4e81b8e79029d9804668: Request end of tablet copy session 835cac8d50ab4eb59fe5537783732606-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.195.189@KRBTEST.COM'} at 127.0.0.1:34592
I20260812 16:52:33.852864 21636 tablet_copy_service.cc:433] P 850b1174ee3e4e81b8e79029d9804668: ending tablet copy session 835cac8d50ab4eb59fe5537783732606-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 835cac8d50ab4eb59fe5537783732606
I20260812 16:52:33.865779 22495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:33.869562 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:33.875295 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:33.882584 22719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:33.885537 22495 server_base.cc:1034] running on GCE node
I20260812 16:52:33.885951 22495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:33.892807 22495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:33.894098 22495 hybrid_clock.cc:648] HybridClock initialized: now 1786553553894025 us; error 56 us; skew 500 ppm
I20260812 16:52:33.894418 22495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:33 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553553, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:33.897527 22495 init.cc:377] Logged in from keytab as oryx/127.8.195.189@KRBTEST.COM (short username oryx)
I20260812 16:52:33.912681 22495 webserver.cc:460] Webserver started at http://127.8.195.189:33589/ using document root <none> and password file <none>
I20260812 16:52:33.913106 22495 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:33.913232 22495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:33.938058 22495 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.002s sys 0.011s
I20260812 16:52:33.940459 22729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:33.941051 22495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 16:52:33.941236 22495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "835cac8d50ab4eb59fe5537783732606"
format_stamp: "Formatted at 2026-08-12 16:52:33 on dist-test-slave-z8x9"
server_key: "f155f5706254bf96780688b79bb6f0b5"
server_key_iv: "10de1b7e732a80c7d9ab7e7779be9184"
server_key_version: "encryptionkey@0"
I20260812 16:52:33.941421 22495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:33.953604 22486 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
I20260812 16:52:33.977596 22495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:33.982717 22495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:34.158041 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.157031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57146)
0812 16:52:34.157517 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.157710 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:46093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260812 16:52:34.159768 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.159059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57158)
0812 16:52:34.159396 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.159506 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:46093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260812 16:52:34.237056 22827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.189:46093 every 8 connection(s)
I20260812 16:52:34.237488 22495 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.189:46093
I20260812 16:52:34.238647 22495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
I20260812 16:52:34.253237 22828 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:52:34.258333 22828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Bootstrap starting.
I20260812 16:52:34.283740 22828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:52:34.284534 22828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: Bootstrap complete.
I20260812 16:52:34.286211 22828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } }
I20260812 16:52:34.286491 22828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 835cac8d50ab4eb59fe5537783732606, State: Initialized, Role: LEARNER
I20260812 16:52:34.286768 22828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } }
I20260812 16:52:34.294476 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } }
I20260812 16:52:34.294863 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:52:34.295944 22828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:34.381017 22854 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260812 16:52:34.429052 22702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:34.429518 22702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:34.429687 22702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:34.444738 22702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:34.444965 22702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:34.445147 22702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:34.445284 22702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:34.445379 22702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:34.461560 22702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.188.keytab
--principal=oryx/127.8.195.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:38463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.195.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:34.463073 22702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:34.464638 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:34.481137 22863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:34.481426 22862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:34.481263 22867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:34.483309 22702 server_base.cc:1034] running on GCE node
I20260812 16:52:34.484444 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:34.486241 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:34.487605 22702 hybrid_clock.cc:648] HybridClock initialized: now 1786553554487516 us; error 79 us; skew 500 ppm
I20260812 16:52:34.488023 22702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553554, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:34.492203 22702 init.cc:377] Logged in from keytab as oryx/127.8.195.188@KRBTEST.COM (short username oryx)
I20260812 16:52:34.494100 22702 webserver.cc:460] Webserver started at http://127.8.195.188:34755/ using document root <none> and password file <none>
I20260812 16:52:34.494828 22702 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:34.494973 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:34.495342 22702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:34.499096 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/instance:
uuid: "bbd740047ff7410386bd06275cf7318b"
format_stamp: "Formatted at 2026-08-12 16:52:34 on dist-test-slave-z8x9"
server_key: "e175da36f8caf25ae53b19c77e4a7589"
server_key_iv: "146343046c15bd7e834098eeaa222100"
server_key_version: "encryptionkey@0"
I20260812 16:52:34.500128 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal/instance:
uuid: "bbd740047ff7410386bd06275cf7318b"
format_stamp: "Formatted at 2026-08-12 16:52:34 on dist-test-slave-z8x9"
server_key: "e175da36f8caf25ae53b19c77e4a7589"
server_key_iv: "146343046c15bd7e834098eeaa222100"
server_key_version: "encryptionkey@0"
I20260812 16:52:34.506398 22702 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 16:52:34.510365 22883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:34.511922 22702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:52:34.512176 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "bbd740047ff7410386bd06275cf7318b"
format_stamp: "Formatted at 2026-08-12 16:52:34 on dist-test-slave-z8x9"
server_key: "e175da36f8caf25ae53b19c77e4a7589"
server_key_iv: "146343046c15bd7e834098eeaa222100"
server_key_version: "encryptionkey@0"
I20260812 16:52:34.512456 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:34.542873 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:34.548542 22702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:34.548928 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:34.575769 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.555292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57172)
0812 16:52:34.555530 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.555554 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.555711 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.556794 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.556800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.557474 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.557716 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.557732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.559968 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.559975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.561604 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.561625 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.562931 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.562940 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.563214 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.564135 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.564161 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.565822 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.571780 (+ 5958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.571787 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.571791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.572573 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.573137 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.573143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.573148 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.573496 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.574488 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.574494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.574740 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.574913 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.575280 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 16:52:34.577236 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.555635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.8.195.189:46093)
0812 16:52:34.556331 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.556339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.556362 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:34.556446 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.556453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.556554 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.557916 (+ 1362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.558197 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.559731 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.561829 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.562678 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.562874 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.566222 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.566232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.566240 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.566288 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.571574 (+ 5286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.572765 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.572771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.572775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.572965 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.573677 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.573685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.573696 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.574237 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.574394 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.576239 (+ 1845us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.576549 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":382,"threads_started":1}
I20260812 16:52:34.579285 22783 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 16:52:34.580094 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850b1174ee3e4e81b8e79029d9804668. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } }
I20260812 16:52:34.580375 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:52:34.587750 22702 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.188:38463
I20260812 16:52:34.587769 22972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.188:38463 every 8 connection(s)
I20260812 16:52:34.590309 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
I20260812 16:52:34.595225 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:34.600570 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 22702
I20260812 16:52:34.604595 22976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553554, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.188@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:34.639652 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.603734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.8.195.188:38463)
0812 16:52:34.605414 (+ 1680us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.605437 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.606556 (+ 1119us) server_negotiation.cc:407] Connection header received
0812 16:52:34.609635 (+ 3079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.609664 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.610162 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.610619 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.615546 (+ 4927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.618671 (+ 3125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.620869 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.622080 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.622299 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.629598 (+ 7299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.629607 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.629619 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.629659 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.633829 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.635165 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.635175 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.635186 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.635497 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.635948 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.635953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.635957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.636676 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.637137 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.637638 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.638114 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1298,"thread_start_us":975,"threads_started":1}
I20260812 16:52:34.648707 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.607921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34604)
0812 16:52:34.625968 (+ 18047us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.626016 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.626193 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.626711 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.626736 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.628103 (+ 1367us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.628789 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.628806 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.631065 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.631071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.633338 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.633364 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.634577 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.634586 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.634805 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.635572 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.635598 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.639805 (+ 4207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.645395 (+ 5590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.645405 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.645410 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.645994 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.646488 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.646492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.646494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.646625 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.647504 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.647511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.647740 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.647942 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.648340 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17773,"thread_start_us":17607,"threads_started":1}
I20260812 16:52:34.649415 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.608019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.8.195.190:35251)
0812 16:52:34.624443 (+ 16424us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.624453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.626323 (+ 1870us) server_negotiation.cc:407] Connection header received
0812 16:52:34.626406 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.626411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.626520 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.626637 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.629030 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.630737 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.633537 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.634288 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.634428 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.640087 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.640094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.640099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.640144 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.645248 (+ 5104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.646151 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.646158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.646161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.646370 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.646786 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.646792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.646795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.647304 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.647512 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.648122 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.648409 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16230,"thread_start_us":16448,"threads_started":1}
I20260812 16:52:34.650930 22976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553554, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.188@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
I20260812 16:52:34.671514 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.652077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.8.195.189:46093)
0812 16:52:34.652268 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.652272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.652507 (+ 235us) server_negotiation.cc:407] Connection header received
0812 16:52:34.652610 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.652616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.652706 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.652823 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.653677 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.655013 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.656385 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.657272 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.657461 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.662228 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.662236 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.662240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.662308 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.667209 (+ 4901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.668074 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.668082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.668085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.668268 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.668814 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.668820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.668823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.670263 (+ 1440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.670457 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.670947 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.671136 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:52:34.671761 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.652065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57184)
0812 16:52:34.652404 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.652420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.652515 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.652873 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.652878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.653345 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.653508 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.653517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.655202 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.655206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.656218 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.656232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.657500 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.657507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.657856 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.658649 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.658668 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.661932 (+ 3264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.667395 (+ 5463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.667401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.667405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.667850 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.668445 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.668452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.668456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.668657 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.670434 (+ 1777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.670441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.670649 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.670848 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.671279 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 16:52:34.673300 22976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553554, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.188@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
I20260812 16:52:34.689303 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.674078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52422)
0812 16:52:34.674319 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.674336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.674463 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.674839 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.674845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.675653 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.675870 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.675882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.676922 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.676929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.678513 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.678529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.679389 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.679397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.679827 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.680565 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.680602 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.683416 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.686631 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.686639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.686642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.687197 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.687586 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.687592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.687596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.687756 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.688269 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.688273 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.688412 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.688556 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.688845 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 16:52:34.689380 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.674314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.8.195.188:38463)
0812 16:52:34.674465 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.674470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.674486 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:52:34.674576 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.674581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.674644 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.674789 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.676042 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.676752 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.678690 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.679147 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.679256 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.683608 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.683613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.683622 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.683650 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.686491 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.687323 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.687331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.687334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.687470 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.687862 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.687869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.687873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.688148 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.688310 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.688640 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.688980 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":54}
I20260812 16:52:34.695575 22976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Bootstrap starting.
I20260812 16:52:34.699628 22976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:34.701406 22976 log.cc:824] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:34.704960 22976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: No bootstrap required, opened a new log
I20260812 16:52:34.713716 22976 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } }
I20260812 16:52:34.714107 22976 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:34.714241 22976 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd740047ff7410386bd06275cf7318b, State: Initialized, Role: FOLLOWER
I20260812 16:52:34.714828 22976 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } }
I20260812 16:52:34.716086 22988 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } } }
I20260812 16:52:34.716702 22988 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:34.717023 22976 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: configured and running, proceeding with master startup.
W20260812 16:52:34.730753 23007 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:52:34.730917 23007 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:52:34.737692 22702 master_runner.cc:428] Detected that this master bbd740047ff7410386bd06275cf7318b is joining an existing cluster
I20260812 16:52:34.737877 22702 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.195.188:38463
I20260812 16:52:34.747036 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21711
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.129@KRBTEST.COM
I20260812 16:52:34.777999 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.744409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.8.195.190:35251)
0812 16:52:34.744603 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.744608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.746120 (+ 1512us) server_negotiation.cc:407] Connection header received
0812 16:52:34.746397 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.746404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.746492 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.746595 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.749342 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.750606 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.755259 (+ 4653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.756140 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.756257 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.768259 (+ 12002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.768267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.768273 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.768327 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.773070 (+ 4743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.774254 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.774262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.774268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.774454 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.775856 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.775862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.775865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.776285 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.776428 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.777356 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.777621 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:52:34.780102 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.744119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34610)
0812 16:52:34.745835 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.745861 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.746117 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.746679 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.746686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.747802 (+ 1116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.748797 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.748818 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.753131 (+ 4313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.753139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.755047 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.755066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.758641 (+ 3575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.758650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.758871 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.759945 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.759981 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.761520 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.773353 (+ 11833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.773360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.773374 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.774082 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.775465 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.775471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.775475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.775706 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.776506 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.776511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.776765 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.776993 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.779257 (+ 2264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"thread_start_us":1312,"threads_started":1}
I20260812 16:52:34.782163 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.748986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.8.195.129:45367)
0812 16:52:34.749814 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.749827 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.749866 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:52:34.749953 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.749965 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.750309 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.750511 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.752950 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.755348 (+ 2398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.758476 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.759772 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.759944 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.768407 (+ 8463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.768439 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.768469 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.768563 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.773857 (+ 5294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.775451 (+ 1594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.775467 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.775473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.775882 (+ 409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.779005 (+ 3123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.779015 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.779021 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.779760 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.780301 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.781213 (+ 912us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.781616 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":3824,"threads_started":1}
I20260812 16:52:34.784118 21963 generic_service.cc:196] Checking for leaks (request via RPC)
W20260812 16:52:34.802191 22892 tablet.cc:2369] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:52:34 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.190@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
I20260812 16:52:34.846998 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.798425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52430)
0812 16:52:34.798942 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.798963 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.799082 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.799933 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.799938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.800614 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.801140 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.801158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.804179 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.804186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.805672 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.805691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.807225 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.807231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.807396 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.808100 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.808127 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.812705 (+ 4578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.840434 (+ 27729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.840443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.840450 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.841190 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.844374 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.844380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.844384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.844584 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.845620 (+ 1036us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.845624 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.845850 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.846075 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.846455 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373}
I20260812 16:52:34.849138 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.744169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.8.195.189:46093)
0812 16:52:34.744355 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.744362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.785257 (+ 40895us) server_negotiation.cc:407] Connection header received
0812 16:52:34.785564 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.785571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.785672 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.785748 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.796174 (+ 10426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.797349 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.805883 (+ 8534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.806893 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.807021 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.837694 (+ 30673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.837703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.837707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.837760 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.843643 (+ 5883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.844749 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.844755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.844759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.844901 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.846056 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.846062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.846066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.846445 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.846596 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.848468 (+ 1872us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.848770 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:52:34.849968 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.746269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57200)
0812 16:52:34.785121 (+ 38852us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.785146 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.785387 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.785981 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.785986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.786906 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.795869 (+ 8963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.795901 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.801666 (+ 5765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.801673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.805382 (+ 3709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.805400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.807771 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.807778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.808015 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.813380 (+ 5365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.813415 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.835563 (+ 22148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.843831 (+ 8268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.843836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.843841 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.844584 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.845712 (+ 1128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.845716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.845720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.845865 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.846974 (+ 1109us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.846979 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.847162 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.848053 (+ 891us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.849634 (+ 1581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35712,"thread_start_us":54530,"threads_started":1}
I20260812 16:52:34.851145 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.798727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.8.195.188:38463)
0812 16:52:34.799331 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.799337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.799354 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:52:34.799460 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.799466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.799556 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.799674 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.801383 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.804009 (+ 2626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.805813 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.806697 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.806836 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.812915 (+ 6079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.812932 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.812939 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.812983 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.840265 (+ 27282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.841687 (+ 1422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.841694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.841699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.844244 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.845031 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.845036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.845039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.845503 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.845665 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.846770 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.847733 (+ 963us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457}
I20260812 16:52:34.852900 21609 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.195.188:38463: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bbd740047ff7410386bd06275cf7318b"
member_type: NON_VOTER
last_known_addr {
host: "127.8.195.188"
port: 38463
}
attrs {
promote: true
}
}
dest_uuid: "850b1174ee3e4e81b8e79029d9804668"
cas_config_opid_index: 5
I20260812 16:52:34.854272 21609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } }
I20260812 16:52:34.856778 22688 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bbd740047ff7410386bd06275cf7318b (127.8.195.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.859570 22930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:34.863656 22930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:52:34.874181 22688 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.195.188:38463
I20260812 16:52:34.874413 23034 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:34.876523 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.808634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52434)
0812 16:52:34.809112 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.809145 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.809350 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.850967 (+ 41617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.850972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.852792 (+ 1820us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.853134 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.853149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.855137 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.855143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.856882 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.856902 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.858144 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.858151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.858322 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.861375 (+ 3053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:34.861407 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:34.863082 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:34.867889 (+ 4807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.867894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.867898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.868456 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.869053 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:34.869059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:34.869063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:34.874304 (+ 5241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:34.875199 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:34.875205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:34.875365 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 16:52:34.875538 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 16:52:34.876049 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260812 16:52:34.877626 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.801423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.8.195.188:38463)
0812 16:52:34.804093 (+ 2670us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.804100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.809290 (+ 5190us) server_negotiation.cc:407] Connection header received
0812 16:52:34.850628 (+ 41338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.850634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.850795 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.851060 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.853432 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.854946 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.857075 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.857796 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.857920 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:34.863435 (+ 5515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:34.863444 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:34.863450 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:34.863490 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:34.867733 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.868738 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.868743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.868747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.868910 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:34.874460 (+ 5550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:34.874465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:34.874469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:34.874960 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:34.876767 (+ 1807us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:34.876935 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 16:52:34.877223 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2504,"thread_start_us":2192,"threads_started":1}
I20260812 16:52:34.888980 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.891156 22783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 16:52:34.892576 22702 master.cc:537] Master@127.8.195.188:38463 shutting down...
I20260812 16:52:34.892130 23030 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260812 16:52:34.894069 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:34.897435 22783 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bbd740047ff7410386bd06275cf7318b (127.8.195.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.902721 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: NON_VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: true } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.903205 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: LEARNER
I20260812 16:52:34.905956 22702 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:34.906379 22702 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:34.906594 22702 tablet_replica.cc:330] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: stopping tablet replica
I20260812 16:52:34.911162 23034 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: attempting to promote NON_VOTER 835cac8d50ab4eb59fe5537783732606 to VOTER
W20260812 16:52:34.911188 21586 consensus_peers.cc:487] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 -> Peer bbd740047ff7410386bd06275cf7318b (127.8.195.188:38463): Couldn't send request to peer bbd740047ff7410386bd06275cf7318b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 16:52:34.912067 23034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } }
I20260812 16:52:34.920033 22783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 LEARNER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 16:52:34.921162 23033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 16:52:34.923987 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.923546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52442)
0812 16:52:34.923704 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.923791 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 16:52:34.924366 21586 consensus_peers.cc:487] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 -> Peer bbd740047ff7410386bd06275cf7318b (127.8.195.188:38463): Couldn't send request to peer bbd740047ff7410386bd06275cf7318b. Status: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:52:34.925062 23030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 835cac8d50ab4eb59fe5537783732606 (127.8.195.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.929508 23034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.929842 23034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:34.930109 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.930451 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:34.930894 22783 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 835cac8d50ab4eb59fe5537783732606 (127.8.195.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.932314 22702 master.cc:552] Master@127.8.195.188:38463 shutdown complete.
I20260812 16:52:34.932394 22702 master_runner.cc:305] Clearing existing system tablet
I20260812 16:52:34.932813 22702 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:34.932898 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:34.935113 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:34.935401 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:34.937758 22702 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260812 16:52:34.940223 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:34.940868 22702 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:34.941421 22702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 16:52:34.941679 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "bbd740047ff7410386bd06275cf7318b"
format_stamp: "Formatted at 2026-08-12 16:52:34 on dist-test-slave-z8x9"
server_key: "e175da36f8caf25ae53b19c77e4a7589"
server_key_iv: "146343046c15bd7e834098eeaa222100"
server_key_version: "encryptionkey@0"
I20260812 16:52:34.947908 22702 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:34.955284 22702 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 16:52:34.955726 22702 log.cc:1196] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 16:52:34.956684 22702 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Deleting consensus metadata
I20260812 16:52:34.957105 22702 master_runner.cc:315] Copying system tablet from 127.8.195.190:35251
I20260812 16:52:34.982898 22702 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: tablet copy: Beginning tablet copy session from remote peer at address 127.8.195.190:35251
I20260812 16:52:34.989355 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21711
I20260812 16:52:35.011430 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.984112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.8.195.190:35251)
0812 16:52:34.984375 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:34.984382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:34.988447 (+ 4065us) server_negotiation.cc:407] Connection header received
0812 16:52:34.988634 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:34.988640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:34.988736 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:34.988859 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:34.994115 (+ 5256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.995800 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.998244 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.999068 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.999199 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.003073 (+ 3874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.003081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.003086 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.003145 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.007640 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.008651 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.008658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.008662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.008825 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.009392 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.009397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.009401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.009808 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.009951 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.010626 (+ 675us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.010917 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260812 16:52:35.012670 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:34.984166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:34618)
0812 16:52:34.988266 (+ 4100us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:34.988292 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:34.988486 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:34.989074 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:34.989079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:34.993446 (+ 4367us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:34.993791 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.993809 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.996077 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.996084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.997953 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:34.997974 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:34.999593 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:34.999602 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:34.999955 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.001326 (+ 1371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:35.001352 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:35.002773 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:35.007870 (+ 5097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.007876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.007881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.008483 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.008986 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.008991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.008995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.009249 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.010049 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:35.010054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:35.010289 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 16:52:35.011835 (+ 1546us) client_negotiation.cc:240] Negotiation successful
0812 16:52:35.012203 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3925,"thread_start_us":3750,"threads_started":1}
I20260812 16:52:35.017532 21636 tablet_copy_service.cc:140] P 850b1174ee3e4e81b8e79029d9804668: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bbd740047ff7410386bd06275cf7318b ({username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:34618)
I20260812 16:52:35.017830 21636 tablet_copy_service.cc:161] P 850b1174ee3e4e81b8e79029d9804668: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bbd740047ff7410386bd06275cf7318b at {username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:34618: session id = bbd740047ff7410386bd06275cf7318b-00000000000000000000000000000000
I20260812 16:52:35.023056 21636 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 16:52:35.025482 22702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 16:52:35.034235 22702 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: tablet copy: Starting download of 0 data blocks...
I20260812 16:52:35.034786 22702 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: tablet copy: Starting download of 1 WAL segments...
I20260812 16:52:35.037851 22702 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 16:52:35.043627 21636 tablet_copy_service.cc:342] P 850b1174ee3e4e81b8e79029d9804668: Request end of tablet copy session bbd740047ff7410386bd06275cf7318b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:34618
I20260812 16:52:35.043800 21636 tablet_copy_service.cc:433] P 850b1174ee3e4e81b8e79029d9804668: ending tablet copy session bbd740047ff7410386bd06275cf7318b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bbd740047ff7410386bd06275cf7318b
I20260812 16:52:35.047024 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:45367
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:35251
--webserver_port=36305
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:52:35.059392 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:35.068730 23058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:35.082598 23063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:35.089319 22702 server_base.cc:1034] running on GCE node
W20260812 16:52:35.090348 23068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:35.091003 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:35.100219 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:35.101593 22702 hybrid_clock.cc:648] HybridClock initialized: now 1786553555101548 us; error 55 us; skew 500 ppm
I20260812 16:52:35.101994 22702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553555, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:35.104580 22702 init.cc:377] Logged in from keytab as oryx/127.8.195.188@KRBTEST.COM (short username oryx)
I20260812 16:52:35.138149 22702 webserver.cc:460] Webserver started at http://127.8.195.188:45093/ using document root <none> and password file <none>
I20260812 16:52:35.138676 22702 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:35.138877 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:35.142692 22702 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 16:52:35.145262 23081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:35.149294 22702 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260812 16:52:35.149549 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "bbd740047ff7410386bd06275cf7318b"
format_stamp: "Formatted at 2026-08-12 16:52:34 on dist-test-slave-z8x9"
server_key: "e175da36f8caf25ae53b19c77e4a7589"
server_key_iv: "146343046c15bd7e834098eeaa222100"
server_key_version: "encryptionkey@0"
I20260812 16:52:35.149842 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:35.195240 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:35.200325 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:35.384011 22854 catalog_manager.cc:1259] Loaded cluster ID: ed653dec459d419f95caf2e8ef0d233a
I20260812 16:52:35.384166 22854 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: loading cluster ID for follower catalog manager: success
I20260812 16:52:35.388572 22854 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: acquiring CA information for follower catalog manager: success
I20260812 16:52:35.392113 22854 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:52:35.395618 22702 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.188:38463
I20260812 16:52:35.395646 23178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.188:38463 every 8 connection(s)
I20260812 16:52:35.396783 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
I20260812 16:52:35.405113 23183 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:52:35.415030 23183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Bootstrap starting.
I20260812 16:52:35.442440 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.421800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52448)
0812 16:52:35.422067 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:35.422087 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:35.422214 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:35.423235 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:35.423240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:35.423871 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:35.424084 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.424096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.427395 (+ 3299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.427403 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.428883 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.428901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.430398 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.430407 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.430612 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.431700 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:35.431725 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:35.432850 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:35.439110 (+ 6260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.439119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.439126 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.439795 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.440371 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.440377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.440382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.440564 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.441325 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:35.441331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:35.441519 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 16:52:35.441688 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 16:52:35.442007 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 16:52:35.443081 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.422053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.8.195.188:38463)
0812 16:52:35.422837 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:35.422843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:35.422871 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:52:35.422946 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:35.422952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:35.423052 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:35.423193 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:35.424280 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.427201 (+ 2921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.429061 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.430197 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.430392 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.433189 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.433197 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.433203 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.433266 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.438922 (+ 5656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.439931 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.439940 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.439946 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.440196 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.440714 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.440719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.440722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.441147 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.441411 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.441808 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.442041 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":441,"threads_started":1}
I20260812 16:52:35.463162 23183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:52:35.463922 23183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Bootstrap complete.
I20260812 16:52:35.465377 23183 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } }
I20260812 16:52:35.465684 23183 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bbd740047ff7410386bd06275cf7318b, State: Initialized, Role: LEARNER
I20260812 16:52:35.465943 23183 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } }
I20260812 16:52:35.467645 23183 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:35.468415 23135 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 16:52:35.474088 23192 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:35.475594 23192 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: LEARNER
I20260812 16:52:35.477701 23192 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850b1174ee3e4e81b8e79029d9804668. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:35.478279 23192 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: LEARNER
I20260812 16:52:35.491802 23218 catalog_manager.cc:1259] Loaded cluster ID: ed653dec459d419f95caf2e8ef0d233a
I20260812 16:52:35.491977 23218 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: loading cluster ID for follower catalog manager: success
I20260812 16:52:35.497890 23218 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: acquiring CA information for follower catalog manager: success
I20260812 16:52:35.502099 23218 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 16:52:35.582108 23057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:35.582473 23057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:35.582654 23057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:35.590631 23057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:35.590843 23057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:35.591038 23057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:35.601847 23057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:45367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=36305
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:35.603322 23057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:35.604678 23057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:35.619668 23228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:35.619745 23226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:35.621102 23225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:35.621362 23057 server_base.cc:1034] running on GCE node
I20260812 16:52:35.622788 23057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:35.625280 23057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:35.626667 23057 hybrid_clock.cc:648] HybridClock initialized: now 1786553555626590 us; error 63 us; skew 500 ppm
I20260812 16:52:35.627338 23057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553555, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:35.634289 23057 init.cc:377] Logged in from keytab as oryx/127.8.195.129@KRBTEST.COM (short username oryx)
I20260812 16:52:35.637349 23057 webserver.cc:460] Webserver started at http://127.8.195.129:36305/ using document root <none> and password file <none>
I20260812 16:52:35.638945 23057 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:35.639317 23057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:35.652016 23057 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.004s
I20260812 16:52:35.658243 23250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:35.661023 23057 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260812 16:52:35.661417 23057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "941321848a1b4dcf9a124db4ad04fb1e"
format_stamp: "Formatted at 2026-08-12 16:52:31 on dist-test-slave-z8x9"
server_key: "59827c98bfb0e1b163c3d721bea50337"
server_key_iv: "bf293eb052d787ec2e2218e2d0f915fe"
server_key_version: "encryptionkey@0"
I20260812 16:52:35.664042 23057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:35.718672 23057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:35.726269 23057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:35.726646 23057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:35.727926 23057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:35.729801 23057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:35.729923 23057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:35.730021 23057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:35.730099 23057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:35.795691 23057 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:45367
I20260812 16:52:35.795892 23480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:45367 every 8 connection(s)
I20260812 16:52:35.797681 23057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:35.804450 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 23057
I20260812 16:52:35.805017 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21990
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.130@KRBTEST.COM
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553555, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553555, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
Aug 12 16:52:35 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553555, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
I20260812 16:52:35.851761 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.806644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.8.195.130:40881)
0812 16:52:35.818160 (+ 11516us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:35.818173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:35.818212 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:52:35.818324 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:35.818333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:35.818675 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:35.818857 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:35.821984 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.823825 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.826634 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.827526 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.836859 (+ 9333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.837212 (+ 353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.837227 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.837246 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.837296 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.843309 (+ 6013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.844417 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.844426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.844429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.844601 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.846429 (+ 1828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.846434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.846437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.849343 (+ 2906us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.850881 (+ 1538us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.851053 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.851351 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11132,"thread_start_us":11797,"threads_started":1}
I20260812 16:52:35.853713 22233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:35.860304 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.802246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:45867 (local address 127.8.195.190:35251)
0812 16:52:35.803466 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:35.803472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:35.816296 (+ 12824us) server_negotiation.cc:407] Connection header received
0812 16:52:35.826116 (+ 9820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:35.826122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:35.826211 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:35.826350 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:35.828589 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.829732 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.834715 (+ 4983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.835411 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.835544 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.846653 (+ 11109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.846660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.846665 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.846708 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.851238 (+ 4530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.854061 (+ 2823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.854073 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.854080 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.854359 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.857380 (+ 3021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.857387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.857391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.857931 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.858184 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.859568 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.859852 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1085,"thread_start_us":956,"threads_started":1}
I20260812 16:52:35.862969 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.802862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:45867)
0812 16:52:35.816035 (+ 13173us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:35.816111 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:35.823983 (+ 7872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:35.826414 (+ 2431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:35.826420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:35.826985 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:35.828113 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.828154 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.830968 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.830978 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.834237 (+ 3259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.834273 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.836829 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.836836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.837251 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.838954 (+ 1703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:35.838991 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:35.846440 (+ 7449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:35.851430 (+ 4990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.851443 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.851463 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.852735 (+ 1272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.856654 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.856661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.856667 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.857155 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.858415 (+ 1260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:35.858425 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:35.859007 (+ 582us) client_negotiation.cc:769] Sending connection context
0812 16:52:35.860662 (+ 1655us) client_negotiation.cc:240] Negotiation successful
0812 16:52:35.861324 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12746,"mutex_wait_us":4,"thread_start_us":12422,"threads_started":1}
I20260812 16:52:35.863750 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.822561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.8.195.129:58241)
0812 16:52:35.833010 (+ 10449us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:35.833041 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:35.833241 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:35.833697 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:35.833702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:35.835187 (+ 1485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:35.835796 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.835811 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.837352 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.837357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.838927 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.838943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.840587 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.840599 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.840937 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.842239 (+ 1302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:35.842271 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:35.851628 (+ 9357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:35.858727 (+ 7099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.858731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.858734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.859314 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.861619 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.861623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.861627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.861826 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.862690 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:35.862700 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:35.862864 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 16:52:35.863033 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 16:52:35.863330 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10288,"mutex_wait_us":1,"thread_start_us":10786,"threads_started":1}
I20260812 16:52:35.864754 23485 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:35.864938 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.821849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:58241 (local address 127.8.195.189:46093)
0812 16:52:35.822571 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:35.822578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:35.833154 (+ 10576us) server_negotiation.cc:407] Connection header received
0812 16:52:35.833412 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:35.833418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:35.833539 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:35.833675 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:35.835999 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.837121 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.839121 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.840038 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.840194 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.852811 (+ 12617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.852820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.852829 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.852888 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.858501 (+ 5613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.861313 (+ 2812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.861319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.861322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.861476 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.861970 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.861977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.861981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.862463 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.862603 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.864296 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.864544 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":435,"threads_started":1}
I20260812 16:52:35.865090 23485 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:35.865856 23491 heartbeater.cc:347] Connected to a master server at 127.8.195.189:46093
I20260812 16:52:35.866106 23491 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:35.866246 23485 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:35.867080 23491 heartbeater.cc:511] Master 127.8.195.189:46093 requested a full tablet report, sending...
I20260812 16:52:35.867273 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.802010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:55813 (local address 127.8.195.188:38463)
0812 16:52:35.802551 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:35.802558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:35.820741 (+ 18183us) server_negotiation.cc:407] Connection header received
0812 16:52:35.824413 (+ 3672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:35.824422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:35.824525 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:35.824588 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:35.829163 (+ 4575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.830692 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.837818 (+ 7126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.838503 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.838614 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.846476 (+ 7862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:35.846488 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:35.846497 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:35.846556 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:35.850498 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.859679 (+ 9181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.859687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.859690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.859875 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:35.860359 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:35.860366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:35.860371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:35.860789 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:35.860937 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:35.866537 (+ 5600us) server_negotiation.cc:299] Negotiation successful
0812 16:52:35.866882 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391}
I20260812 16:52:35.868748 21609 ts_manager.cc:194] Re-registered known tserver with Master: 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:35.869441 22766 ts_manager.cc:194] Registered new tserver with Master: 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:35.870011 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:45867
I20260812 16:52:35.871594 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:35.806073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.8.195.129:55813)
0812 16:52:35.820484 (+ 14411us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:35.820506 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:35.823983 (+ 3477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:35.824989 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:35.824998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:35.826649 (+ 1651us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:35.828895 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.828914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.834604 (+ 5690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.834609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.836530 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:35.836544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:35.841569 (+ 5025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:35.841584 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:35.841880 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:35.842657 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:35.842686 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:35.846133 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:35.851465 (+ 5332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.851469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.851472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.859508 (+ 8036us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.860039 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:35.860046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:35.860051 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:35.860211 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:35.861018 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:35.861022 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:35.861158 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 16:52:35.870772 (+ 9614us) client_negotiation.cc:240] Negotiation successful
0812 16:52:35.871136 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14234,"mutex_wait_us":1221,"spinlock_wait_cycles":15360,"thread_start_us":14744,"threads_started":1}
I20260812 16:52:35.872987 23484 heartbeater.cc:347] Connected to a master server at 127.8.195.188:38463
I20260812 16:52:35.873291 23484 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:35.873787 23484 heartbeater.cc:511] Master 127.8.195.188:38463 requested a full tablet report, sending...
I20260812 16:52:35.875869 23114 ts_manager.cc:194] Registered new tserver with Master: 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:35.939824 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: NON_VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: true } } }
I20260812 16:52:35.940416 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:35.950697 23033 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: attempting to promote NON_VOTER bbd740047ff7410386bd06275cf7318b to VOTER
I20260812 16:52:35.951548 23033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } }
I20260812 16:52:35.954288 22783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 16:52:35.954510 23135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 LEARNER]: Refusing update from remote peer 850b1174ee3e4e81b8e79029d9804668: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 16:52:35.955160 23033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 16:52:35.955597 23510 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 16:52:35.963052 23510 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bbd740047ff7410386bd06275cf7318b (127.8.195.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.964634 23135 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bbd740047ff7410386bd06275cf7318b (127.8.195.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.968461 23510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.968845 23510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:35.969233 23512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.969672 23512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:35.972995 23192 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.973446 23192 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:35.979007 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.979326 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:35.979944 22783 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bbd740047ff7410386bd06275cf7318b (127.8.195.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.984054 23518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "850b1174ee3e4e81b8e79029d9804668" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:35.984678 23518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:35.988013 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21990
I20260812 16:52:36.015992 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:40881
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:35251
--webserver_port=42173
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:36.406023 23520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:36.406416 23520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:36.406558 23520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:36.415256 23520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:36.415386 23520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:36.415508 23520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:36.428313 23520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:40881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=42173
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:36.429885 23520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:36.431511 23520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:36.447157 23530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:36.447244 23540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:36.447158 23532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:36.448180 23520 server_base.cc:1034] running on GCE node
I20260812 16:52:36.449023 23520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:36.450152 23520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:36.451401 23520 hybrid_clock.cc:648] HybridClock initialized: now 1786553556451354 us; error 58 us; skew 500 ppm
I20260812 16:52:36.451804 23520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:36 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:36.455324 23520 init.cc:377] Logged in from keytab as oryx/127.8.195.130@KRBTEST.COM (short username oryx)
I20260812 16:52:36.456949 23520 webserver.cc:460] Webserver started at http://127.8.195.130:42173/ using document root <none> and password file <none>
I20260812 16:52:36.457836 23520 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:36.458029 23520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:36.464133 23520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 16:52:36.466946 23553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:36.468485 23520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:52:36.468717 23520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "043a4c55c2e4a2f36e578a551670cd71"
server_key_iv: "4334c566ca994ff5ab2b52d3903e7d65"
server_key_version: "encryptionkey@0"
I20260812 16:52:36.469827 23520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:36.484329 23520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:36.489717 23520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:36.490049 23520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:36.491165 23520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:36.492738 23520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:36.492839 23520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:36.492980 23520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:36.493068 23520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:36.530834 23520 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40881
I20260812 16:52:36.530901 23758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40881 every 8 connection(s)
I20260812 16:52:36.532485 23520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:36.539746 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 23520
I20260812 16:52:36.540125 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 22253
Aug 12 16:52:36 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
Aug 12 16:52:36 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
Aug 12 16:52:36 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
Aug 12 16:52:36 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.131@KRBTEST.COM
I20260812 16:52:36.583879 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.537451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:60463 (local address 127.8.195.190:35251)
0812 16:52:36.549984 (+ 12533us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:36.551212 (+ 1228us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:36.551317 (+ 105us) server_negotiation.cc:407] Connection header received
0812 16:52:36.551404 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:36.551413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:36.551518 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:36.551636 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:36.558998 (+ 7362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.560810 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.563913 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.564573 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.564680 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.572972 (+ 8292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:36.572978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:36.572981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:36.573137 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:36.577676 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.578738 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.578744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.578748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.578963 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.580955 (+ 1992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.580959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.580961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.581344 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:36.581495 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:36.583210 (+ 1715us) server_negotiation.cc:299] Negotiation successful
0812 16:52:36.583468 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12321,"thread_start_us":12644,"threads_started":1}
I20260812 16:52:36.584229 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.535620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:51227 (local address 127.8.195.188:38463)
0812 16:52:36.536206 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:36.536215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:36.537480 (+ 1265us) server_negotiation.cc:407] Connection header received
0812 16:52:36.550392 (+ 12912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:36.550398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:36.550517 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:36.550657 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:36.557747 (+ 7090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.559522 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.563318 (+ 3796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.564231 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.564361 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.572729 (+ 8368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:36.572737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:36.572742 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:36.572790 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:36.577662 (+ 4872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.579553 (+ 1891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.579562 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.579569 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.579798 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.580339 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.580348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.580355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.580780 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:36.580923 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:36.583124 (+ 2201us) server_negotiation.cc:299] Negotiation successful
0812 16:52:36.583449 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":297,"threads_started":1}
I20260812 16:52:36.586221 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.536283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.8.195.130:51227)
0812 16:52:36.537292 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:36.537357 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:36.548752 (+ 11395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:36.551468 (+ 2716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:36.551486 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:36.552578 (+ 1092us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:36.557429 (+ 4851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.557472 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.559842 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.559852 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.563006 (+ 3154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.563021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.564765 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.564774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.565124 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.566106 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:36.566157 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:36.570818 (+ 4661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:36.578798 (+ 7980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.578803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.578807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.579367 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.579943 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.579951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.579958 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.580182 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.581877 (+ 1695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:36.581889 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:36.582675 (+ 786us) client_negotiation.cc:769] Sending connection context
0812 16:52:36.584486 (+ 1811us) client_negotiation.cc:240] Negotiation successful
0812 16:52:36.584971 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":145,"spinlock_wait_cycles":44416,"thread_start_us":1152,"threads_started":1}
I20260812 16:52:36.587046 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.539182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:55545 (local address 127.8.195.189:46093)
0812 16:52:36.555643 (+ 16461us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:36.555649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:36.561424 (+ 5775us) server_negotiation.cc:407] Connection header received
0812 16:52:36.561929 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:36.561935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:36.562028 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:36.562105 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:36.563888 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.565366 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.567791 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.568438 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.568606 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.576597 (+ 7991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:36.576607 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:36.576615 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:36.576709 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:36.580816 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.581843 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.581848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.581852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.582023 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.582517 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.582522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.582525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.582902 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:36.583030 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:36.586259 (+ 3229us) server_negotiation.cc:299] Negotiation successful
0812 16:52:36.586538 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16296,"thread_start_us":16190,"threads_started":1}
I20260812 16:52:36.588092 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.538790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.8.195.130:55545)
0812 16:52:36.561283 (+ 22493us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:36.561322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:36.561749 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:36.562318 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:36.562323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:36.563038 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:36.563696 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.563711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.565642 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.565651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.567550 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.567569 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.568682 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.568690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.568956 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.569551 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:36.569582 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:36.576274 (+ 6692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:36.580990 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.580999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.581006 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.581703 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.582176 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.582181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.582184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.582384 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.584158 (+ 1774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:36.584163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:36.584326 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 16:52:36.584474 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 16:52:36.584799 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22297,"thread_start_us":22664,"threads_started":1}
I20260812 16:52:36.589061 23761 heartbeater.cc:347] Connected to a master server at 127.8.195.189:46093
I20260812 16:52:36.589465 23761 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:36.589897 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.537318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:60463)
0812 16:52:36.550781 (+ 13463us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:36.550818 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:36.551000 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:36.551907 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:36.551915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:36.552774 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:36.558656 (+ 5882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.558677 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.561299 (+ 2622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.561305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.563130 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:36.563151 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.566928 (+ 3777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.566943 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:36.567128 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.568075 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:36.568099 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:36.572655 (+ 4556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:36.577894 (+ 5239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.577909 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.577930 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.578572 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.579527 (+ 955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:36.579535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:36.579541 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:36.579854 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:36.582380 (+ 2526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:36.582385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:36.582596 (+ 211us) client_negotiation.cc:769] Sending connection context
0812 16:52:36.582973 (+ 377us) client_negotiation.cc:240] Negotiation successful
0812 16:52:36.583528 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13266,"mutex_wait_us":4365,"thread_start_us":13052,"threads_started":1}
I20260812 16:52:36.591190 23759 heartbeater.cc:347] Connected to a master server at 127.8.195.188:38463
I20260812 16:52:36.591362 23759 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:36.591359 23761 heartbeater.cc:511] Master 127.8.195.189:46093 requested a full tablet report, sending...
I20260812 16:52:36.592303 23759 heartbeater.cc:511] Master 127.8.195.188:38463 requested a full tablet report, sending...
I20260812 16:52:36.592914 23760 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:36.593026 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:36.541444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.8.195.131:35483)
0812 16:52:36.553945 (+ 12501us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:36.553957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:36.556430 (+ 2473us) server_negotiation.cc:407] Connection header received
0812 16:52:36.556815 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:36.556834 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:36.557225 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:36.557424 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:36.560896 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.562661 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.564867 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:36.565990 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:36.566089 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:36.584259 (+ 18170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:36.584281 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:36.584300 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:36.584345 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:36.588700 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.589900 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.589912 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.589919 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.590109 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:36.590556 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:36.590563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:36.590565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:36.591010 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:36.591346 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:36.591984 (+ 638us) server_negotiation.cc:299] Negotiation successful
0812 16:52:36.592357 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12298,"thread_start_us":12496,"threads_started":1}
I20260812 16:52:36.593116 23760 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:36.593598 23760 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:36.594677 23114 ts_manager.cc:194] Registered new tserver with Master: f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:36.595407 22477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:36.601788 22766 ts_manager.cc:194] Registered new tserver with Master: f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:36.603415 21609 ts_manager.cc:194] Re-registered known tserver with Master: f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:36.604805 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:60463
I20260812 16:52:36.689931 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 22253
I20260812 16:52:36.719981 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:35483
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:35251
--webserver_port=36869
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:52:36.874480 23485 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
W20260812 16:52:37.108000 23781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:37.108359 23781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:37.108532 23781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:37.116544 23781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:37.116722 23781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:37.116899 23781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:37.131098 23781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=36869
--tserver_master_addrs=127.8.195.190:35251,127.8.195.189:46093,127.8.195.188:38463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:37.132831 23781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:37.134248 23781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:37.150277 23788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:37.150275 23791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:37.151003 23787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:37.151286 23781 server_base.cc:1034] running on GCE node
I20260812 16:52:37.152096 23781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:37.153381 23781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:37.154666 23781 hybrid_clock.cc:648] HybridClock initialized: now 1786553557154605 us; error 70 us; skew 500 ppm
I20260812 16:52:37.155102 23781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:37 dist-test-slave-z8x9 krb5kdc[21523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:37.162147 23781 init.cc:377] Logged in from keytab as oryx/127.8.195.131@KRBTEST.COM (short username oryx)
I20260812 16:52:37.163875 23781 webserver.cc:460] Webserver started at http://127.8.195.131:36869/ using document root <none> and password file <none>
I20260812 16:52:37.164623 23781 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:37.164779 23781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:37.171705 23781 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:37.175124 23805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:37.177150 23781 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:52:37.177459 23781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "b1030e057604403ab2db3cb7898edb1c"
format_stamp: "Formatted at 2026-08-12 16:52:32 on dist-test-slave-z8x9"
server_key: "3a8dc5cc211f529b4e16ea01a5ae87c2"
server_key_iv: "ea50f8a2735e14a22bd2909d2450c72a"
server_key_version: "encryptionkey@0"
I20260812 16:52:37.178824 23781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:37.202673 23781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:37.208398 23781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:37.208724 23781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:37.210314 23781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:37.212195 23781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:37.212329 23781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:37.212486 23781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:37.212592 23781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:37.281997 23781 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:35483
I20260812 16:52:37.282048 24014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:35483 every 8 connection(s)
I20260812 16:52:37.284679 23781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:37.289964 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 23781
Aug 12 16:52:37 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
Aug 12 16:52:37 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.188@KRBTEST.COM
Aug 12 16:52:37 dist-test-slave-z8x9 krb5kdc[21523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.189@KRBTEST.COM
I20260812 16:52:37.416114 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.340065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.8.195.189:46093)
0812 16:52:37.389900 (+ 49835us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.389907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.389927 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:52:37.389984 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.389990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.390069 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.390172 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.393557 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.396156 (+ 2599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.399630 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.400895 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.401051 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.406024 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.406034 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.406043 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.406094 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.410894 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.412040 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.412050 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.412056 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.412260 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.413157 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.413163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.413189 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.413610 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.415194 (+ 1584us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.415333 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.415607 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49659,"thread_start_us":49467,"threads_started":1}
I20260812 16:52:37.417191 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.289071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:58201 (local address 127.8.195.188:38463)
0812 16:52:37.326520 (+ 37449us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.326530 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.326565 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:52:37.326657 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.326665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.326788 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.326940 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.345318 (+ 18378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.346647 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.361209 (+ 14562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.362155 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.362292 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.376112 (+ 13820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.376119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.376123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.376160 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.386642 (+ 10482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.410119 (+ 23477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.410128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.410131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.410357 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.413741 (+ 3384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.413749 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.413753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.414177 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.414402 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.416379 (+ 1977us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.416638 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37230,"thread_start_us":37980,"threads_started":1}
I20260812 16:52:37.418304 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.288691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:35497 (local address 127.8.195.190:35251)
0812 16:52:37.331774 (+ 43083us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.331784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.331814 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:52:37.331895 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.331903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.332015 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.332160 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.344984 (+ 12824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.346333 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.358414 (+ 12081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.359407 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.359546 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.377854 (+ 18308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.377862 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.377866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.377934 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.388072 (+ 10138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.408621 (+ 20549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.408629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.408634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.408878 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.415087 (+ 6209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.415093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.415095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.415524 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.415675 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.417595 (+ 1920us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.417894 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42875,"thread_start_us":41929,"threads_started":1}
I20260812 16:52:37.420539 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.340080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.8.195.190:35251)
0812 16:52:37.375064 (+ 34984us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.375188 (+ 124us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.375214 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:52:37.375309 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.375318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.375438 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.382222 (+ 6784us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.382491 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.384084 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.387042 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.391059 (+ 4017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.391176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.401515 (+ 10339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.401525 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.401532 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.401587 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.406303 (+ 4716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.410735 (+ 4432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.410740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.410743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.410917 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.412370 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.412395 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.412398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.418152 (+ 5754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.418436 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.419593 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.419885 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34805,"thread_start_us":43172,"threads_started":1}
I20260812 16:52:37.420650 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.289561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.8.195.131:58201)
0812 16:52:37.314980 (+ 25419us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.315050 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.318460 (+ 3410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.333985 (+ 15525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.333991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.341046 (+ 7055us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:37.345063 (+ 4017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.345078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.346882 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.346887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.353229 (+ 6342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.353249 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.362655 (+ 9406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.362664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.362929 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.367778 (+ 4849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:37.367811 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:37.372881 (+ 5070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:37.387265 (+ 14384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.387279 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.387302 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.388186 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.410954 (+ 22768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.410961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.410967 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.411243 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.414515 (+ 3272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:37.414537 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:37.414811 (+ 274us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.418545 (+ 3734us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.419200 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23788,"thread_start_us":30500,"threads_started":1}
I20260812 16:52:37.422410 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.289987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:35497)
0812 16:52:37.319991 (+ 30004us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.320018 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.320174 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.333499 (+ 13325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.333519 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.335904 (+ 2385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:37.344749 (+ 8845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.344796 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.346537 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.346543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.355721 (+ 9178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.355750 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.360019 (+ 4269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.360028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.360464 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.365437 (+ 4973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:37.365487 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:37.375788 (+ 10301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:37.401637 (+ 25849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.401646 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.401653 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.402462 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.409526 (+ 7064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.409533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.409536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.409898 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.415801 (+ 5903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:37.415808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:37.416006 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.421406 (+ 5400us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.421844 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29809,"spinlock_wait_cycles":1001216,"thread_start_us":23646,"threads_started":1}
I20260812 16:52:37.423132 24017 heartbeater.cc:347] Connected to a master server at 127.8.195.188:38463
I20260812 16:52:37.423563 24017 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:37.423627 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.321028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:60433 (local address 127.8.195.189:46093)
0812 16:52:37.333427 (+ 12399us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.333433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.338494 (+ 5061us) server_negotiation.cc:407] Connection header received
0812 16:52:37.341559 (+ 3065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.341569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.341690 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.341799 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.350114 (+ 8315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.351968 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.363433 (+ 11465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.364313 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.364440 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.402713 (+ 38273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.402733 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.402739 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.402789 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.407467 (+ 4678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.409101 (+ 1634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.409107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.409110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.409311 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.411394 (+ 2083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.411399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.411402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.412023 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.412240 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.422835 (+ 10595us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.423176 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12235,"thread_start_us":12050,"threads_started":1}
I20260812 16:52:37.424561 24017 heartbeater.cc:511] Master 127.8.195.188:38463 requested a full tablet report, sending...
I20260812 16:52:37.424621 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.327059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.8.195.131:60433)
0812 16:52:37.338362 (+ 11303us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.338386 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.340098 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.342113 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.342122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.344212 (+ 2090us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:37.347425 (+ 3213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.347447 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.352281 (+ 4834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.352287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.358094 (+ 5807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.358121 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.364792 (+ 6671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.364811 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.364989 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.368643 (+ 3654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:37.368675 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:37.375174 (+ 6499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:37.407711 (+ 32537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.407732 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.407739 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.408402 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.410528 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:37.410533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:37.410536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:37.410796 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:37.412409 (+ 1613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:37.412420 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:37.413184 (+ 764us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.423888 (+ 10704us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.424204 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11146,"mutex_wait_us":1888,"thread_start_us":12690,"threads_started":1}
I20260812 16:52:37.425089 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.353565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.8.195.188:38463)
0812 16:52:37.364679 (+ 11114us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.364686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.375762 (+ 11076us) server_negotiation.cc:407] Connection header received
0812 16:52:37.376058 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.376064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.376143 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.376242 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.391656 (+ 15414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.393125 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.396112 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.397254 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.397361 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.411385 (+ 14024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.411394 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.411399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.411441 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.417936 (+ 6495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.421869 (+ 3933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.421879 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.421886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.422125 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.422609 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.422617 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.422624 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.423025 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.423285 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.424315 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.424586 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10928,"thread_start_us":11114,"threads_started":1}
I20260812 16:52:37.425840 24025 heartbeater.cc:347] Connected to a master server at 127.8.195.189:46093
I20260812 16:52:37.425971 24025 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:37.426188 24018 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35251
I20260812 16:52:37.426345 24018 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:37.426576 24025 heartbeater.cc:511] Master 127.8.195.189:46093 requested a full tablet report, sending...
I20260812 16:52:37.427168 24018 heartbeater.cc:511] Master 127.8.195.190:35251 requested a full tablet report, sending...
I20260812 16:52:37.427771 23114 ts_manager.cc:194] Registered new tserver with Master: b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483)
I20260812 16:52:37.428480 22766 ts_manager.cc:194] Registered new tserver with Master: b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483)
I20260812 16:52:37.428915 21609 ts_manager.cc:194] Re-registered known tserver with Master: b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483)
I20260812 16:52:37.430298 21609 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:35497
I20260812 16:52:37.435532 21607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:37.444446 21607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:37.497891 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.476220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45367 (local address 127.0.0.1:59322)
0812 16:52:37.485096 (+ 8876us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.485126 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.485353 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.486926 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.486934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.486982 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:37.487403 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.487419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.490353 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.490362 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.493493 (+ 3131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.493514 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.496562 (+ 3048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.496572 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.496837 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.496958 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.497040 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.497279 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8660,"mutex_wait_us":43,"thread_start_us":8395,"threads_started":1}
I20260812 16:52:37.500218 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.477465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.8.195.129:45367)
0812 16:52:37.486037 (+ 8572us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.486045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.486120 (+ 75us) server_negotiation.cc:407] Connection header received
0812 16:52:37.486264 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.486276 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.486602 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.487634 (+ 1032us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:37.487977 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.490168 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.493698 (+ 3530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.496264 (+ 2566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.499094 (+ 2830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.499444 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.499526 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.499762 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8395,"thread_start_us":8270,"threads_started":1}
I20260812 16:52:37.500463 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.477194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.8.195.131:35483)
0812 16:52:37.483497 (+ 6303us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.483509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.486571 (+ 3062us) server_negotiation.cc:407] Connection header received
0812 16:52:37.486911 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.486922 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.487196 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.487401 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:37.488568 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.490752 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.495384 (+ 4632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.498147 (+ 2763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.498288 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.498692 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.499593 (+ 901us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.499874 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5307,"thread_start_us":4941,"threads_started":1}
I20260812 16:52:37.501159 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.476607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:35483 (local address 127.0.0.1:40030)
0812 16:52:37.486443 (+ 9836us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.486465 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.486695 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.487858 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.487865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.487899 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:37.488310 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.488324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.490980 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.490987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.495091 (+ 4104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.495113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.498976 (+ 3863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.498982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.499333 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.499367 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.500706 (+ 1339us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.500899 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9614,"thread_start_us":9459,"threads_started":1}
I20260812 16:52:37.505282 23355 tablet_service.cc:1467] Processing CreateTablet for tablet 88d4794efbc948b4a5a1c25edb338371 (DEFAULT_TABLE table=test-table [id=969a2d8ec2974db492bb0a45bcd63161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:37.505264 23898 tablet_service.cc:1467] Processing CreateTablet for tablet 88d4794efbc948b4a5a1c25edb338371 (DEFAULT_TABLE table=test-table [id=969a2d8ec2974db492bb0a45bcd63161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:37.507037 23355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d4794efbc948b4a5a1c25edb338371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:37.507228 23898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d4794efbc948b4a5a1c25edb338371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:37.513674 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.490310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40881 (local address 127.0.0.1:50168)
0812 16:52:37.498717 (+ 8407us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.498739 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.498884 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.502247 (+ 3363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.502252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.502281 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:37.502514 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.502526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.506786 (+ 4260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.506792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.509283 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.509306 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.512734 (+ 3428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.512741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.512957 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.513012 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.513094 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.513280 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8267,"thread_start_us":8497,"threads_started":1}
I20260812 16:52:37.516762 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.486752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.8.195.130:40881)
0812 16:52:37.495213 (+ 8461us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.495223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.501495 (+ 6272us) server_negotiation.cc:407] Connection header received
0812 16:52:37.501628 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.501636 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.501980 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.502718 (+ 738us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:37.503394 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.506532 (+ 3138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.510514 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.512423 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.515714 (+ 3291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.516004 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.516108 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.516312 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8166,"thread_start_us":8006,"threads_started":1}
I20260812 16:52:37.520495 23633 tablet_service.cc:1467] Processing CreateTablet for tablet 88d4794efbc948b4a5a1c25edb338371 (DEFAULT_TABLE table=test-table [id=969a2d8ec2974db492bb0a45bcd63161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:37.522557 23633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d4794efbc948b4a5a1c25edb338371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:37.528926 24092 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Bootstrap starting.
I20260812 16:52:37.541965 24092 tablet_bootstrap.cc:654] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:37.544227 24092 log.cc:824] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:37.546641 24093 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Bootstrap starting.
I20260812 16:52:37.551133 24097 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Bootstrap starting.
I20260812 16:52:37.552129 24093 tablet_bootstrap.cc:654] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:37.553834 24093 log.cc:824] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:37.560209 24097 tablet_bootstrap.cc:654] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:37.564317 24097 log.cc:824] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:37.564793 24092 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: No bootstrap required, opened a new log
I20260812 16:52:37.564850 24093 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: No bootstrap required, opened a new log
I20260812 16:52:37.565289 24092 ts_tablet_manager.cc:1397] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Time spent bootstrapping tablet: real 0.037s user 0.018s sys 0.008s
I20260812 16:52:37.565364 24093 ts_tablet_manager.cc:1397] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.004s
I20260812 16:52:37.579679 24093 raft_consensus.cc:348] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.580265 24093 raft_consensus.cc:374] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:37.580384 24093 raft_consensus.cc:729] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941321848a1b4dcf9a124db4ad04fb1e, State: Initialized, Role: FOLLOWER
I20260812 16:52:37.581864 24093 consensus_queue.cc:260] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.582485 24092 raft_consensus.cc:348] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.583047 24092 raft_consensus.cc:374] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:37.583177 24092 raft_consensus.cc:729] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1030e057604403ab2db3cb7898edb1c, State: Initialized, Role: FOLLOWER
I20260812 16:52:37.584311 24097 tablet_bootstrap.cc:492] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: No bootstrap required, opened a new log
I20260812 16:52:37.584127 24092 consensus_queue.cc:260] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.584774 24097 ts_tablet_manager.cc:1397] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.018s
I20260812 16:52:37.599622 24093 ts_tablet_manager.cc:1428] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Time spent starting tablet: real 0.034s user 0.023s sys 0.008s
I20260812 16:52:37.600569 24018 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
I20260812 16:52:37.604911 24092 ts_tablet_manager.cc:1428] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Time spent starting tablet: real 0.039s user 0.016s sys 0.018s
I20260812 16:52:37.604938 24097 raft_consensus.cc:348] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.605571 24097 raft_consensus.cc:374] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:37.605719 24097 raft_consensus.cc:729] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2dd765a2a044d8bbfdc86ac2c21a04e, State: Initialized, Role: FOLLOWER
I20260812 16:52:37.607074 24097 consensus_queue.cc:260] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.608027 23760 heartbeater.cc:503] Master 127.8.195.190:35251 was elected leader, sending a full tablet report...
I20260812 16:52:37.611320 24097 ts_tablet_manager.cc:1428] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Time spent starting tablet: real 0.026s user 0.011s sys 0.006s
I20260812 16:52:37.703910 24103 raft_consensus.cc:482] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:37.704316 24103 raft_consensus.cc:504] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.706425 24103 leader_election.cc:288] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367), f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:37.714857 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.707653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:40803 (local address 127.8.195.130:40881)
0812 16:52:37.707988 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.707995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.708016 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:52:37.708089 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.708094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.708172 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.708272 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:37.709126 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.710358 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.712255 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.713506 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.713711 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.713797 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.714338 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.714552 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 16:52:37.714921 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.706749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40881 (local address 127.8.195.131:40803)
0812 16:52:37.707190 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.707210 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.707723 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.708537 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.708543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.708571 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:37.708959 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.708971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.710516 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.710522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.712035 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.712050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.713707 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.713723 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.713998 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.714058 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.714174 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.714383 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":1,"spinlock_wait_cycles":36736}
I20260812 16:52:37.715404 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.706972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:45689 (local address 127.8.195.129:45367)
0812 16:52:37.707185 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.707192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.707546 (+ 354us) server_negotiation.cc:407] Connection header received
0812 16:52:37.707661 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.707666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.707753 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.707849 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:37.708582 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.709852 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.712565 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.713959 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.714158 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.714229 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.715009 (+ 780us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.715162 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:52:37.715925 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.706910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45367 (local address 127.8.195.131:45689)
0812 16:52:37.707183 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:37.707204 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:37.707386 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:37.708069 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:37.708074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:37.708101 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:37.708324 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.708338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.710073 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.710080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.712348 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:37.712368 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.714166 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.714172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:37.714533 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.714568 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:52:37.714671 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:52:37.714830 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"spinlock_wait_cycles":2816}
I20260812 16:52:37.716604 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d4794efbc948b4a5a1c25edb338371" candidate_uuid: "b1030e057604403ab2db3cb7898edb1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" is_pre_election: true
I20260812 16:52:37.717301 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d4794efbc948b4a5a1c25edb338371" candidate_uuid: "b1030e057604403ab2db3cb7898edb1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941321848a1b4dcf9a124db4ad04fb1e" is_pre_election: true
I20260812 16:52:37.717628 23671 raft_consensus.cc:2454] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1030e057604403ab2db3cb7898edb1c in term 0.
I20260812 16:52:37.717846 23391 raft_consensus.cc:2454] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1030e057604403ab2db3cb7898edb1c in term 0.
I20260812 16:52:37.718492 23815 leader_election.cc:302] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1030e057604403ab2db3cb7898edb1c, f2dd765a2a044d8bbfdc86ac2c21a04e; no voters:
I20260812 16:52:37.719058 24103 raft_consensus.cc:2781] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:37.719259 24103 raft_consensus.cc:482] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:37.719417 24103 raft_consensus.cc:3037] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:37.722177 24103 raft_consensus.cc:504] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.723188 24103 leader_election.cc:288] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [CANDIDATE]: Term 1 election: Requested vote from peers 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367), f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881)
I20260812 16:52:37.723599 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d4794efbc948b4a5a1c25edb338371" candidate_uuid: "b1030e057604403ab2db3cb7898edb1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941321848a1b4dcf9a124db4ad04fb1e"
I20260812 16:52:37.723605 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d4794efbc948b4a5a1c25edb338371" candidate_uuid: "b1030e057604403ab2db3cb7898edb1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e"
I20260812 16:52:37.723839 23391 raft_consensus.cc:3037] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:37.723872 23671 raft_consensus.cc:3037] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:37.726763 23391 raft_consensus.cc:2454] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1030e057604403ab2db3cb7898edb1c in term 1.
I20260812 16:52:37.727300 23810 leader_election.cc:302] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 941321848a1b4dcf9a124db4ad04fb1e, b1030e057604403ab2db3cb7898edb1c; no voters:
I20260812 16:52:37.727743 24103 raft_consensus.cc:2781] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:37.727954 23671 raft_consensus.cc:2454] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1030e057604403ab2db3cb7898edb1c in term 1.
I20260812 16:52:37.728453 24103 raft_consensus.cc:686] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 1 LEADER]: Becoming Leader. State: Replica: b1030e057604403ab2db3cb7898edb1c, State: Running, Role: LEADER
I20260812 16:52:37.729072 24103 consensus_queue.cc:237] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } }
I20260812 16:52:37.735666 21609 catalog_manager.cc:5495] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c reported cstate change: term changed from 0 to 1, leader changed from <none> to b1030e057604403ab2db3cb7898edb1c (127.8.195.131). New cstate: current_term: 1 leader_uuid: "b1030e057604403ab2db3cb7898edb1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } health_report { overall_health: UNKNOWN } } }
I20260812 16:52:37.770798 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.764142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.8.195.131:35483)
0812 16:52:37.764385 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.764392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.764415 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:37.764613 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.764617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.764748 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.764937 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:37.765505 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.766255 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.767805 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.768664 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.768818 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.769567 (+ 749us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:37.770144 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:37.770324 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.770417 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.770506 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:52:37.781306 23391 raft_consensus.cc:1261] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Refusing update from remote peer b1030e057604403ab2db3cb7898edb1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:37.781303 23671 raft_consensus.cc:1261] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Refusing update from remote peer b1030e057604403ab2db3cb7898edb1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:37.782829 24105 consensus_queue.cc:1035] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:37.783373 24103 consensus_queue.cc:1035] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:52:37.800776 23763 tablet.cc:2369] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:52:37.810787 24112 mvcc.cc:205] Tried to move back new op lower bound from 7317723372658647040 to 7317723372463394816. Current Snapshot: MvccSnapshot[applied={T|T < 7317723372658647040}]
I20260812 16:52:37.814488 24109 mvcc.cc:205] Tried to move back new op lower bound from 7317723372658647040 to 7317723372463394816. Current Snapshot: MvccSnapshot[applied={T|T < 7317723372658647040 or (T in {7317723372658647040})}]
I20260812 16:52:37.845902 21607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:37.846232 21607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:37.855376 21607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 850b1174ee3e4e81b8e79029d9804668: Sending DeleteTablet for 3 replicas of tablet 88d4794efbc948b4a5a1c25edb338371
I20260812 16:52:37.856984 23355 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d4794efbc948b4a5a1c25edb338371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:37 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:59322
I20260812 16:52:37.857282 23898 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d4794efbc948b4a5a1c25edb338371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:37 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:40030
I20260812 16:52:37.857602 23633 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d4794efbc948b4a5a1c25edb338371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:37 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:50168
I20260812 16:52:37.858708 24129 tablet_replica.cc:330] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: stopping tablet replica
I20260812 16:52:37.859321 24129 raft_consensus.cc:2229] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:37.869076 24129 raft_consensus.cc:2258] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:37.874825 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21549
I20260812 16:52:37.876255 21656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:37.897161 24135 tablet_replica.cc:330] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: stopping tablet replica
I20260812 16:52:37.898108 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.860892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.8.195.189:46093)
0812 16:52:37.861139 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.861149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.871526 (+ 10377us) server_negotiation.cc:407] Connection header received
0812 16:52:37.872946 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.872954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.873099 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.873211 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.879462 (+ 6251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.880841 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.883882 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.884696 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.884845 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.889587 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.889604 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.889610 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.889674 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.893678 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.894676 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.894682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.894686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.894947 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.895522 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.895528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.895531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.896145 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.896476 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.897256 (+ 780us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.897723 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:52:37.898206 24135 raft_consensus.cc:2229] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:37.898965 24135 raft_consensus.cc:2258] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:37.901949 24129 ts_tablet_manager.cc:1905] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:37.916183 24129 ts_tablet_manager.cc:1918] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:37.916419 24129 log.cc:1196] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/88d4794efbc948b4a5a1c25edb338371
I20260812 16:52:37.917162 24129 ts_tablet_manager.cc:1939] T 88d4794efbc948b4a5a1c25edb338371 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting consensus metadata
I20260812 16:52:37.919819 24135 ts_tablet_manager.cc:1905] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:37.922322 24131 tablet_replica.cc:330] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: stopping tablet replica
I20260812 16:52:37.923017 24131 raft_consensus.cc:2229] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:37.923787 24131 raft_consensus.cc:2258] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:37.927142 24135 ts_tablet_manager.cc:1918] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:37.927424 24135 log.cc:1196] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/88d4794efbc948b4a5a1c25edb338371
I20260812 16:52:37.928246 24135 ts_tablet_manager.cc:1939] T 88d4794efbc948b4a5a1c25edb338371 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting consensus metadata
I20260812 16:52:37.932036 24131 ts_tablet_manager.cc:1905] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:37.941098 24131 ts_tablet_manager.cc:1918] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:37.941356 24131 log.cc:1196] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/88d4794efbc948b4a5a1c25edb338371
I20260812 16:52:37.942297 24131 ts_tablet_manager.cc:1939] T 88d4794efbc948b4a5a1c25edb338371 P b1030e057604403ab2db3cb7898edb1c: Deleting consensus metadata
I20260812 16:52:37.946218 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:37.873897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.8.195.188:38463)
0812 16:52:37.874116 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:37.874122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:37.899718 (+ 25596us) server_negotiation.cc:407] Connection header received
0812 16:52:37.901934 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:37.901941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:37.902102 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:37.902186 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:37.904713 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.906006 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.908869 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:37.909753 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:37.909902 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:37.919319 (+ 9417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:37.919328 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:37.919332 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:37.919386 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:37.933330 (+ 13944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.937246 (+ 3916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.937257 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.937264 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.937550 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:37.941013 (+ 3463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:37.941022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:37.941028 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:37.942073 (+ 1045us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:37.942368 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:37.945514 (+ 3146us) server_negotiation.cc:299] Negotiation successful
0812 16:52:37.945782 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:52:38.095911 21583 catalog_manager.cc:4841] TS 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367): tablet 88d4794efbc948b4a5a1c25edb338371 (table test-table [id=969a2d8ec2974db492bb0a45bcd63161]) successfully deleted
I20260812 16:52:38.096364 21586 catalog_manager.cc:4841] TS b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483): tablet 88d4794efbc948b4a5a1c25edb338371 (table test-table [id=969a2d8ec2974db492bb0a45bcd63161]) successfully deleted
I20260812 16:52:38.096643 21586 catalog_manager.cc:4841] TS f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881): tablet 88d4794efbc948b4a5a1c25edb338371 (table test-table [id=969a2d8ec2974db492bb0a45bcd63161]) successfully deleted
I20260812 16:52:38.097786 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21549
I20260812 16:52:38.191363 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:38.168786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.8.195.189:46093)
0812 16:52:38.169012 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:38.169019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:38.170434 (+ 1415us) server_negotiation.cc:407] Connection header received
0812 16:52:38.170687 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:38.170694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:38.170788 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:38.170894 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:38.173416 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:38.174626 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:38.176825 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:38.177505 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:38.177614 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:38.183169 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:38.183177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:38.183182 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:38.183252 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:38.187449 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:38.188408 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:38.188416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:38.188420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:38.188612 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:38.189142 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:38.189151 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:38.189155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:38.189705 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:38.189944 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:38.190644 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 16:52:38.190880 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 16:52:38.191581 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:38.171362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.8.195.188:38463)
0812 16:52:38.171572 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:38.171580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:38.171905 (+ 325us) server_negotiation.cc:407] Connection header received
0812 16:52:38.172185 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:38.172194 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:38.172288 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:38.172368 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:38.175367 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:38.176711 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:38.179173 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:38.179816 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:38.179952 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:38.184330 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:38.184338 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:38.184343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:38.184389 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:38.188125 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:38.189036 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:38.189044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:38.189050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:38.189244 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:38.189697 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:38.189703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:38.189706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:38.190089 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:38.190330 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:38.190971 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 16:52:38.191240 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:52:38.827872 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:38.826947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:47993)
0812 16:52:38.827504 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:38.827637 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":317,"threads_started":1}
W20260812 16:52:38.828486 24018 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:35251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:38.841003 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:38.837493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:51065)
0812 16:52:38.840585 (+ 3092us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:38.840805 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2963,"threads_started":1}
W20260812 16:52:38.841768 23760 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:35251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:38.847297 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:38.846420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:42377)
0812 16:52:38.846912 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:38.847074 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":247,"threads_started":1}
W20260812 16:52:38.848122 23485 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:35251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.356020 24156 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 850b1174ee3e4e81b8e79029d9804668)
I20260812 16:52:39.356341 24156 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } }
I20260812 16:52:39.357887 24156 leader_election.cc:288] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251), 835cac8d50ab4eb59fe5537783732606 (127.8.195.189:46093)
I20260812 16:52:39.359804 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.358284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35006)
0812 16:52:39.359354 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.359580 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":314,"threads_started":1}
W20260812 16:52:39.360095 23082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.361790 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.361397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35012)
0812 16:52:39.361563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.361646 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:52:39.361997 23082 leader_election.cc:334] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251): Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.362233 24162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 850b1174ee3e4e81b8e79029d9804668)
I20260812 16:52:39.362465 24162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } }
I20260812 16:52:39.363322 24162 leader_election.cc:288] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251), bbd740047ff7410386bd06275cf7318b (127.8.195.188:38463)
I20260812 16:52:39.364539 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.363388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35014)
0812 16:52:39.363997 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.364206 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":301,"threads_started":1}
W20260812 16:52:39.364866 22733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.367166 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.358137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:46093 (local address 127.0.0.1:57234)
0812 16:52:39.358953 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.358986 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.359173 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.359938 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.359942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.359969 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.360343 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.360366 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.361573 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.361577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.363469 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.363489 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.365874 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.365884 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.366167 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.366254 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.366391 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.366601 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":858,"threads_started":1}
I20260812 16:52:39.368145 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.358296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.8.195.189:46093)
0812 16:52:39.359615 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.359619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.359635 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:52:39.359697 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.359701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.359771 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.359917 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.360526 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.361394 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.363688 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.365603 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.365833 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.366115 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.367494 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.367662 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1222,"thread_start_us":1044,"threads_started":1}
I20260812 16:52:39.368923 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd740047ff7410386bd06275cf7318b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "835cac8d50ab4eb59fe5537783732606" is_pre_election: true
I20260812 16:52:39.369388 22783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd740047ff7410386bd06275cf7318b in term 1.
I20260812 16:52:39.369998 23088 leader_election.cc:302] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 835cac8d50ab4eb59fe5537783732606, bbd740047ff7410386bd06275cf7318b; no voters: 850b1174ee3e4e81b8e79029d9804668
I20260812 16:52:39.370410 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.369627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35028)
0812 16:52:39.369929 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.370154 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260812 16:52:39.370465 24156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 16:52:39.370594 24156 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 850b1174ee3e4e81b8e79029d9804668)
I20260812 16:52:39.370680 24156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 1 FOLLOWER]: Advancing to term 2
W20260812 16:52:39.370637 22733 leader_election.cc:334] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251): Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.373112 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.363995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.8.195.188:38463)
0812 16:52:39.364542 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.364547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.365085 (+ 538us) server_negotiation.cc:407] Connection header received
0812 16:52:39.365154 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.365162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.365269 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.365386 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.366760 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.368467 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.371089 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.372035 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.372145 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.372311 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.372652 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.372752 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":283,"threads_started":1}
I20260812 16:52:39.373229 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.363577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38463 (local address 127.0.0.1:52488)
0812 16:52:39.364781 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.364800 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.364955 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.365678 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.365686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.365726 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.366484 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.366501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.368712 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.368719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.370884 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.370902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.372230 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.372236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.372448 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.372489 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.372629 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.372789 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"mutex_wait_us":393,"thread_start_us":5794,"threads_started":1}
I20260812 16:52:39.373756 24156 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } }
I20260812 16:52:39.373867 23135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "835cac8d50ab4eb59fe5537783732606" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbd740047ff7410386bd06275cf7318b" is_pre_election: true
I20260812 16:52:39.374306 23135 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 835cac8d50ab4eb59fe5537783732606 in current term 2: Already voted for candidate bbd740047ff7410386bd06275cf7318b in this term.
I20260812 16:52:39.374702 24156 leader_election.cc:288] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 election: Requested vote from peers 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251), 835cac8d50ab4eb59fe5537783732606 (127.8.195.189:46093)
I20260812 16:52:39.374979 22738 leader_election.cc:302] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 835cac8d50ab4eb59fe5537783732606; no voters: 850b1174ee3e4e81b8e79029d9804668, bbd740047ff7410386bd06275cf7318b
I20260812 16:52:39.375198 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.374746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35036)
0812 16:52:39.374947 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.375014 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 16:52:39.375305 24162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:52:39.375221 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbd740047ff7410386bd06275cf7318b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "835cac8d50ab4eb59fe5537783732606"
I20260812 16:52:39.376338 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.375942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35052)
0812 16:52:39.376104 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.376196 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:52:39.376600 23082 leader_election.cc:334] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251): Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
I20260812 16:52:39.378024 24162 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:52:39.380314 22783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd740047ff7410386bd06275cf7318b in term 2.
I20260812 16:52:39.381388 23088 leader_election.cc:302] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 835cac8d50ab4eb59fe5537783732606, bbd740047ff7410386bd06275cf7318b; no voters: 850b1174ee3e4e81b8e79029d9804668
I20260812 16:52:39.381738 24156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:52:39.382318 24156 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [term 2 LEADER]: Becoming Leader. State: Replica: bbd740047ff7410386bd06275cf7318b, State: Running, Role: LEADER
I20260812 16:52:39.382769 24156 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } }
I20260812 16:52:39.384826 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd740047ff7410386bd06275cf7318b. Latest consensus state: current_term: 2 leader_uuid: "bbd740047ff7410386bd06275cf7318b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:39.385267 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:39.385926 24183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:39.390010 24183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:39.390974 24183 catalog_manager.cc:1259] Loaded cluster ID: ed653dec459d419f95caf2e8ef0d233a
I20260812 16:52:39.391103 24183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:39.392381 24183 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:39.393371 24183 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Loaded TSK: 0
I20260812 16:52:39.395205 24183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:52:39.450183 23112 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:39.457787 23112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:39.462463 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.461958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35074)
0812 16:52:39.462147 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.462252 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 16:52:39.462473 22783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [term 2 FOLLOWER]: Refusing update from remote peer bbd740047ff7410386bd06275cf7318b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 16:52:39.463320 24175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [LEADER]: Connected to new peer: Peer: permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 16:52:39.463632 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.463171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35078)
0812 16:52:39.463396 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.463468 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:52:39.463821 23082 consensus_peers.cc:487] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b -> Peer 850b1174ee3e4e81b8e79029d9804668 (127.8.195.190:35251): Couldn't send request to peer 850b1174ee3e4e81b8e79029d9804668. Status: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 16:52:39.466428 24162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd740047ff7410386bd06275cf7318b. Latest consensus state: current_term: 2 leader_uuid: "bbd740047ff7410386bd06275cf7318b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:39.466789 24162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:39.468367 24156 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bbd740047ff7410386bd06275cf7318b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:39.468748 24156 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:39.469626 24162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bbd740047ff7410386bd06275cf7318b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "850b1174ee3e4e81b8e79029d9804668" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35251 } } peers { permanent_uuid: "835cac8d50ab4eb59fe5537783732606" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 46093 } attrs { promote: false } } peers { permanent_uuid: "bbd740047ff7410386bd06275cf7318b" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38463 } attrs { promote: false } } }
I20260812 16:52:39.469936 24162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835cac8d50ab4eb59fe5537783732606 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:52:39.487665 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.478267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:35483 (local address 127.0.0.1:40056)
0812 16:52:39.478425 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.478447 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.478561 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.479360 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.479365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.479396 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.479596 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.479609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.481448 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.481458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.484210 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.484266 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.486721 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.486732 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.487114 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.487177 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.487268 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.487401 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":4}
I20260812 16:52:39.488658 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.478429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.8.195.131:35483)
0812 16:52:39.479020 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.479028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.479048 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:52:39.479123 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.479130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.479230 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.479335 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.479788 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.481252 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.484561 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.486477 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.488105 (+ 1628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.488246 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.488310 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.488431 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":337,"threads_started":1}
I20260812 16:52:39.489488 23898 tablet_service.cc:1467] Processing CreateTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 (DEFAULT_TABLE table=test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:39.490039 23898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e62cef0c7fd461cb7f591a207295a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:39.493799 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.478624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40881 (local address 127.0.0.1:50180)
0812 16:52:39.478966 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.478984 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.479103 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.487689 (+ 8586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.487697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.487731 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.488137 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.488154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.489651 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.489656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.491719 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.491736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.492946 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.492956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.493127 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.493152 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.493307 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.493463 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":3}
I20260812 16:52:39.493798 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.479347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.8.195.130:40881)
0812 16:52:39.487399 (+ 8052us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.487404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.487420 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:52:39.487477 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.487481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.487535 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.487618 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.488320 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.489494 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.491894 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.492695 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.492881 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.493007 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.493365 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.493516 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7936,"thread_start_us":7816,"threads_started":1}
I20260812 16:52:39.495399 23633 tablet_service.cc:1467] Processing CreateTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 (DEFAULT_TABLE table=test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:39.495918 23633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e62cef0c7fd461cb7f591a207295a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:39.496940 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.479627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.8.195.129:45367)
0812 16:52:39.485754 (+ 6127us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.485763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.486534 (+ 771us) server_negotiation.cc:407] Connection header received
0812 16:52:39.486617 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.486622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.486700 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.486800 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.488396 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.489495 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.494213 (+ 4718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.495505 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.495611 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.495724 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.496395 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.496562 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5956,"thread_start_us":5838,"threads_started":1}
I20260812 16:52:39.496949 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.479144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45367 (local address 127.0.0.1:59330)
0812 16:52:39.486248 (+ 7104us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.486273 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.486417 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.486992 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.486997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.487024 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.488170 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.488180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.491917 (+ 3737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.491922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.493988 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.494006 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.495983 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.495989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.496194 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.496232 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.496331 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.496477 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6973,"mutex_wait_us":670,"thread_start_us":6783,"threads_started":1}
I20260812 16:52:39.497915 24203 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Bootstrap starting.
I20260812 16:52:39.499416 24203 tablet_bootstrap.cc:654] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:39.501781 24203 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: No bootstrap required, opened a new log
I20260812 16:52:39.501959 24203 ts_tablet_manager.cc:1397] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 16:52:39.502242 23355 tablet_service.cc:1467] Processing CreateTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 (DEFAULT_TABLE table=test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:39.502713 23355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e62cef0c7fd461cb7f591a207295a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:39.502830 24203 raft_consensus.cc:348] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.503103 24203 raft_consensus.cc:374] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:39.503226 24203 raft_consensus.cc:729] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1030e057604403ab2db3cb7898edb1c, State: Initialized, Role: FOLLOWER
I20260812 16:52:39.503502 24203 consensus_queue.cc:260] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.504406 24203 ts_tablet_manager.cc:1428] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 16:52:39.509230 24017 heartbeater.cc:503] Master 127.8.195.188:38463 was elected leader, sending a full tablet report...
I20260812 16:52:39.510746 24209 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Bootstrap starting.
I20260812 16:52:39.512687 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.505761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:43533)
0812 16:52:39.512011 (+ 6250us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.512289 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6048,"thread_start_us":5875,"threads_started":1}
I20260812 16:52:39.512702 24209 tablet_bootstrap.cc:654] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:39.514184 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.513653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:59103)
0812 16:52:39.513878 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.513998 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 16:52:39.515425 24209 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: No bootstrap required, opened a new log
I20260812 16:52:39.515604 24209 ts_tablet_manager.cc:1397] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 16:52:39.516306 24209 raft_consensus.cc:348] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.516520 24209 raft_consensus.cc:374] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:39.516624 24209 raft_consensus.cc:729] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2dd765a2a044d8bbfdc86ac2c21a04e, State: Initialized, Role: FOLLOWER
I20260812 16:52:39.516916 24209 consensus_queue.cc:260] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.517738 24214 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Bootstrap starting.
I20260812 16:52:39.517841 24209 ts_tablet_manager.cc:1428] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:52:39.519795 24214 tablet_bootstrap.cc:654] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:39.520920 23759 heartbeater.cc:503] Master 127.8.195.188:38463 was elected leader, sending a full tablet report...
I20260812 16:52:39.521934 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.518406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:56675)
0812 16:52:39.520591 (+ 2185us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.520808 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2035,"thread_start_us":1831,"threads_started":1}
I20260812 16:52:39.522547 24214 tablet_bootstrap.cc:492] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: No bootstrap required, opened a new log
I20260812 16:52:39.522706 24214 ts_tablet_manager.cc:1397] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 16:52:39.523339 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.522793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:45879)
0812 16:52:39.523102 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.523192 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260812 16:52:39.523622 24214 raft_consensus.cc:348] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.523831 24214 raft_consensus.cc:374] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:39.523901 24214 raft_consensus.cc:729] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941321848a1b4dcf9a124db4ad04fb1e, State: Initialized, Role: FOLLOWER
I20260812 16:52:39.524106 24214 consensus_queue.cc:260] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.524824 24214 ts_tablet_manager.cc:1428] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 16:52:39.526407 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.525268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:52203)
0812 16:52:39.525898 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.526082 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":328,"threads_started":1}
I20260812 16:52:39.527359 23484 heartbeater.cc:503] Master 127.8.195.188:38463 was elected leader, sending a full tablet report...
I20260812 16:52:39.528045 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.527251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:50927)
0812 16:52:39.527421 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.527524 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 16:52:39.573279 23493 tablet.cc:2369] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:52:39.597230 24030 tablet.cc:2369] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:52:39.734712 24217 raft_consensus.cc:482] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:39.734948 24217 raft_consensus.cc:504] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.736567 24217 leader_election.cc:288] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483), 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:39.743944 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.736916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:55563 (local address 127.8.195.131:35483)
0812 16:52:39.737187 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.737195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.737537 (+ 342us) server_negotiation.cc:407] Connection header received
0812 16:52:39.737784 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.737792 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.737884 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.737977 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.738594 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.739405 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.741416 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.742722 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.742908 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.742986 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.743528 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.743669 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:52:39.743943 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.737038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:50615 (local address 127.8.195.129:45367)
0812 16:52:39.737357 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.737365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.737859 (+ 494us) server_negotiation.cc:407] Connection header received
0812 16:52:39.738032 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.738038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.738112 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.738215 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:39.738804 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.739572 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.741458 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.742722 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.742907 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.742986 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.743528 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.743669 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:52:39.744382 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.736821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:35483 (local address 127.8.195.130:55563)
0812 16:52:39.737342 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.737409 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.737615 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.738032 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.738038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.738066 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.738422 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.738436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.739551 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.739556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.741136 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.741149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.743059 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.743065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.743308 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.743375 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.743482 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.743641 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":39}
I20260812 16:52:39.744412 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.736880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45367 (local address 127.8.195.130:50615)
0812 16:52:39.737722 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.737740 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:39.737885 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:39.738247 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:39.738252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:39.738279 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:39.738634 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.738648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.739718 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.739723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.741197 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:39.741240 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.743027 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.743040 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:39.743302 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.743384 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 16:52:39.743482 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 16:52:39.743641 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":66,"thread_start_us":494,"threads_started":1}
I20260812 16:52:39.745241 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e62cef0c7fd461cb7f591a207295a23" candidate_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941321848a1b4dcf9a124db4ad04fb1e" is_pre_election: true
I20260812 16:52:39.745317 23933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e62cef0c7fd461cb7f591a207295a23" candidate_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1030e057604403ab2db3cb7898edb1c" is_pre_election: true
I20260812 16:52:39.745520 23391 raft_consensus.cc:2454] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2dd765a2a044d8bbfdc86ac2c21a04e in term 0.
I20260812 16:52:39.745800 23933 raft_consensus.cc:2454] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2dd765a2a044d8bbfdc86ac2c21a04e in term 0.
I20260812 16:52:39.746148 23555 leader_election.cc:302] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 941321848a1b4dcf9a124db4ad04fb1e, f2dd765a2a044d8bbfdc86ac2c21a04e; no voters:
I20260812 16:52:39.746659 24217 raft_consensus.cc:2781] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:39.746776 24217 raft_consensus.cc:482] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:39.746893 24217 raft_consensus.cc:3037] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:39.749390 24217 raft_consensus.cc:504] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.750167 24217 leader_election.cc:288] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [CANDIDATE]: Term 1 election: Requested vote from peers b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483), 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367)
I20260812 16:52:39.750720 23933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e62cef0c7fd461cb7f591a207295a23" candidate_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1030e057604403ab2db3cb7898edb1c"
I20260812 16:52:39.750962 23933 raft_consensus.cc:3037] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:39.750888 23391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e62cef0c7fd461cb7f591a207295a23" candidate_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941321848a1b4dcf9a124db4ad04fb1e"
I20260812 16:52:39.751143 23391 raft_consensus.cc:3037] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:39.753247 23933 raft_consensus.cc:2454] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2dd765a2a044d8bbfdc86ac2c21a04e in term 1.
I20260812 16:52:39.753340 23391 raft_consensus.cc:2454] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2dd765a2a044d8bbfdc86ac2c21a04e in term 1.
I20260812 16:52:39.753786 23557 leader_election.cc:302] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1030e057604403ab2db3cb7898edb1c, f2dd765a2a044d8bbfdc86ac2c21a04e; no voters:
I20260812 16:52:39.754124 24217 raft_consensus.cc:2781] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:39.754604 24217 raft_consensus.cc:686] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 LEADER]: Becoming Leader. State: Replica: f2dd765a2a044d8bbfdc86ac2c21a04e, State: Running, Role: LEADER
I20260812 16:52:39.755113 24217 consensus_queue.cc:237] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } }
I20260812 16:52:39.755689 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.755174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:36697)
0812 16:52:39.755379 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.755515 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 16:52:39.759272 23112 catalog_manager.cc:5495] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e reported cstate change: term changed from 0 to 1, leader changed from <none> to f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130). New cstate: current_term: 1 leader_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2dd765a2a044d8bbfdc86ac2c21a04e" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 } health_report { overall_health: UNKNOWN } } }
I20260812 16:52:39.796836 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.791305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.8.195.130:40881)
0812 16:52:39.791547 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.791553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.791577 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:52:39.791704 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.791708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.791790 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.791970 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:39.792346 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.793094 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.794469 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.794908 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.795055 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.795537 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:39.796112 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:39.796291 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.796350 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.796438 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 16:52:39.801963 23391 raft_consensus.cc:1261] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Refusing update from remote peer f2dd765a2a044d8bbfdc86ac2c21a04e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:39.802340 23933 raft_consensus.cc:1261] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Refusing update from remote peer f2dd765a2a044d8bbfdc86ac2c21a04e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:39.802896 24227 consensus_queue.cc:1035] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [LEADER]: Connected to new peer: Peer: permanent_uuid: "941321848a1b4dcf9a124db4ad04fb1e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:39.803402 24217 consensus_queue.cc:1035] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1030e057604403ab2db3cb7898edb1c" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:39.807204 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.806546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:54469)
0812 16:52:39.806752 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.806928 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 16:52:39.807644 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.807178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:34799)
0812 16:52:39.807375 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.807494 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 16:52:39.818835 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.818084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.131:58999)
0812 16:52:39.818460 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.818590 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260812 16:52:39.819794 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.819020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.129:53785)
0812 16:52:39.819410 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.819537 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260812 16:52:39.823590 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.822704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.8.195.130:42929)
0812 16:52:39.823285 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.823406 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429}
I20260812 16:52:39.838160 23112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:39.838447 23112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:39.844674 23112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbd740047ff7410386bd06275cf7318b: Sending DeleteTablet for 3 replicas of tablet 7e62cef0c7fd461cb7f591a207295a23
I20260812 16:52:39.845884 23898 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:39 UTC) from {username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:40056
I20260812 16:52:39.846441 23355 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:39 UTC) from {username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:59330
I20260812 16:52:39.846660 24247 tablet_replica.cc:330] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: stopping tablet replica
I20260812 16:52:39.846899 24247 raft_consensus.cc:2229] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:39.847252 24248 tablet_replica.cc:330] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: stopping tablet replica
I20260812 16:52:39.847254 24247 raft_consensus.cc:2258] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:39.847522 24248 raft_consensus.cc:2229] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:39.847939 24248 raft_consensus.cc:2258] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:39.848332 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 23057
I20260812 16:52:39.848783 24247 ts_tablet_manager.cc:1905] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:39.851686 23633 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e62cef0c7fd461cb7f591a207295a23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:39 UTC) from {username='oryx', principal='oryx/127.8.195.188@KRBTEST.COM'} at 127.0.0.1:50180
I20260812 16:52:39.856841 24247 ts_tablet_manager.cc:1918] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:39.857023 24247 log.cc:1196] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/7e62cef0c7fd461cb7f591a207295a23
I20260812 16:52:39.857626 24247 ts_tablet_manager.cc:1939] T 7e62cef0c7fd461cb7f591a207295a23 P b1030e057604403ab2db3cb7898edb1c: Deleting consensus metadata
I20260812 16:52:39.857683 24253 tablet_replica.cc:330] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: stopping tablet replica
I20260812 16:52:39.857940 24253 raft_consensus.cc:2229] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:39.858523 24248 ts_tablet_manager.cc:1905] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:39.858651 24253 raft_consensus.cc:2258] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:39.858906 23088 catalog_manager.cc:4841] TS b1030e057604403ab2db3cb7898edb1c (127.8.195.131:35483): tablet 7e62cef0c7fd461cb7f591a207295a23 (table test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]) successfully deleted
I20260812 16:52:39.861701 24253 ts_tablet_manager.cc:1905] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:39.864792 24248 ts_tablet_manager.cc:1918] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:39.865007 24248 log.cc:1196] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/7e62cef0c7fd461cb7f591a207295a23
I20260812 16:52:39.865592 24248 ts_tablet_manager.cc:1939] T 7e62cef0c7fd461cb7f591a207295a23 P 941321848a1b4dcf9a124db4ad04fb1e: Deleting consensus metadata
I20260812 16:52:39.866911 23084 catalog_manager.cc:4841] TS 941321848a1b4dcf9a124db4ad04fb1e (127.8.195.129:45367): tablet 7e62cef0c7fd461cb7f591a207295a23 (table test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]) successfully deleted
I20260812 16:52:39.868538 24253 ts_tablet_manager.cc:1918] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:39.868682 24253 log.cc:1196] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/7e62cef0c7fd461cb7f591a207295a23
I20260812 16:52:39.869252 24253 ts_tablet_manager.cc:1939] T 7e62cef0c7fd461cb7f591a207295a23 P f2dd765a2a044d8bbfdc86ac2c21a04e: Deleting consensus metadata
I20260812 16:52:39.869812 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.869242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35094)
0812 16:52:39.869439 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.869582 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 16:52:39.870424 23088 catalog_manager.cc:4841] TS f2dd765a2a044d8bbfdc86ac2c21a04e (127.8.195.130:40881): tablet 7e62cef0c7fd461cb7f591a207295a23 (table test-table [id=00b7ba0ccb684eb5a31e0cb110c0c163]) successfully deleted
I20260812 16:52:39.871155 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.870799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35100)
0812 16:52:39.870963 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:39.871031 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 16:52:39.872395 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:39.849760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.8.195.129:45367)
0812 16:52:39.849971 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:39.849980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:39.850592 (+ 612us) server_negotiation.cc:407] Connection header received
0812 16:52:39.850932 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:39.850939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:39.851027 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:39.851134 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:39.853538 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.854846 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.856952 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:39.857948 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:39.858074 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:39.862750 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:39.862780 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:39.862801 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:39.862876 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:39.867640 (+ 4764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:39.869121 (+ 1481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:39.869134 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:39.869139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:39.869603 (+ 464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:39.870187 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:39.870193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:39.870197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:39.870953 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:39.871280 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:39.871922 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 16:52:39.872094 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 16:52:39.873356 23466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:39.971481 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 23057
I20260812 16:52:40.004302 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 23520
I20260812 16:52:40.022359 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:40.005150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.8.195.130:40881)
0812 16:52:40.005456 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:40.005463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:40.005509 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:52:40.005774 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:40.005780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:40.005878 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:40.005994 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:40.007328 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:40.008526 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:40.010562 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:40.010986 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:40.011084 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:40.014620 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:40.014643 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:40.014661 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:40.014712 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:40.018672 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:40.019691 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:40.019699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:40.019701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:40.019869 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:40.020464 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:40.020474 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:40.020477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:40.021108 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:40.021460 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:40.021810 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 16:52:40.022007 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260812 16:52:40.023193 23752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:40.145725 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 23520
I20260812 16:52:40.178399 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 23781
I20260812 16:52:40.193121 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:40.179315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.8.195.131:35483)
0812 16:52:40.179511 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:40.179519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:40.179607 (+ 88us) server_negotiation.cc:407] Connection header received
0812 16:52:40.179832 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:40.179836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:40.179936 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:40.180051 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:40.181545 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:40.182344 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:40.183621 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:40.184046 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:40.184210 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:40.187205 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:40.187222 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:40.187240 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:40.187276 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:40.190457 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:40.191173 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:40.191181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:40.191183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:40.191312 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:40.191650 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:40.191654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:40.191656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:40.192047 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:40.192280 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:40.192612 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 16:52:40.192796 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:52:40.194368 24004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:40.319621 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 23781
I20260812 16:52:40.349646 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 22495
I20260812 16:52:40.350885 22819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:40.437760 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:40.436905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35102)
0812 16:52:40.437481 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:40.437579 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":334,"threads_started":1}
I20260812 16:52:40.439191 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:40.438805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35251 (local address 127.0.0.1:35106)
0812 16:52:40.438983 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:40.439072 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:35251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 16:52:40.471638 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 22495
I20260812 16:52:40.504956 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 22702
I20260812 16:52:40.506134 23168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:40.629729 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 22702
2026-08-12T16:52:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11661 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:40 dist-test-slave-z8x9 krb5kdc[24272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:40 dist-test-slave-z8x9 krb5kdc[24272](info): set up 2 sockets
Aug 12 16:52:40 dist-test-slave-z8x9 krb5kdc[24272](info): commencing operation
krb5kdc: starting...
W20260812 16:52:42.043568 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:42 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab.
Entry for principal oryx/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab.
I20260812 16:52:42.333629 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35181
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab
--principal=oryx/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:42.680492 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:42.680805 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:42.680970 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:42.688556 24300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:42.688681 24300 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 16:52:42.688755 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:42.688853 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:42.688946 24300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:42.689050 24300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:42.699960 24300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35181
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.190.keytab
--principal=oryx/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:42.701736 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:42.703127 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:42.716293 24307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:42.716302 24309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:42.716958 24313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:42.717793 24300 server_base.cc:1034] running on GCE node
I20260812 16:52:42.718732 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:42.720260 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:42.721681 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786553562721575 us; error 46 us; skew 500 ppm
I20260812 16:52:42.722141 24300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:42 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553562, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:42.729666 24300 init.cc:377] Logged in from keytab as oryx/127.8.195.190@KRBTEST.COM (short username oryx)
I20260812 16:52:42.731627 24300 webserver.cc:460] Webserver started at http://127.8.195.190:40353/ using document root <none> and password file <none>
I20260812 16:52:42.732532 24300 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:42.732710 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:42.733104 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:42.736538 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "9ccc996b6474479f8aa040d507c5bee7"
format_stamp: "Formatted at 2026-08-12 16:52:42 on dist-test-slave-z8x9"
server_key: "b0a096599acfd0231312be26398feb0d"
server_key_iv: "3e95881afa1d7f592ebed22782a2552f"
server_key_version: "encryptionkey@0"
I20260812 16:52:42.737607 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "9ccc996b6474479f8aa040d507c5bee7"
format_stamp: "Formatted at 2026-08-12 16:52:42 on dist-test-slave-z8x9"
server_key: "b0a096599acfd0231312be26398feb0d"
server_key_iv: "3e95881afa1d7f592ebed22782a2552f"
server_key_version: "encryptionkey@0"
I20260812 16:52:42.744302 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260812 16:52:42.748633 24330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:42.750907 24300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:52:42.751153 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "9ccc996b6474479f8aa040d507c5bee7"
format_stamp: "Formatted at 2026-08-12 16:52:42 on dist-test-slave-z8x9"
server_key: "b0a096599acfd0231312be26398feb0d"
server_key_iv: "3e95881afa1d7f592ebed22782a2552f"
server_key_version: "encryptionkey@0"
I20260812 16:52:42.751464 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:42.770958 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:42.775704 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:42.775946 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:42.808449 24300 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:35181
I20260812 16:52:42.808470 24436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:35181 every 8 connection(s)
I20260812 16:52:42.810043 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:42.813862 24437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:42.815842 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 24300
I20260812 16:52:42.816068 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:42.816753 8974 external_mini_cluster.cc:1428] Setting key 9a8abc73b0e5fa093938940c13a5c127
I20260812 16:52:42.826177 24437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Bootstrap starting.
I20260812 16:52:42.829561 24437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:42.830674 24437 log.cc:824] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:52:42 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:42.833832 24437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: No bootstrap required, opened a new log
I20260812 16:52:42.840380 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:42.819044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.8.195.190:35181)
0812 16:52:42.819941 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:42.819954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:42.820000 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:52:42.821859 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:42.821885 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:42.822314 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:42.822721 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:42.824346 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:42.825798 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:42.827170 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:42.827899 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:42.828112 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:42.832548 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:42.832557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:42.832574 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:42.832613 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:42.835931 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:42.836688 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:42.836706 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:42.836712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:42.836903 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:42.837409 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:42.837416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:42.837427 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:42.837985 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:42.838353 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:42.838859 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 16:52:42.839302 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":418,"threads_started":1}
I20260812 16:52:42.842192 24437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } }
I20260812 16:52:42.842592 24437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:42.842717 24437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccc996b6474479f8aa040d507c5bee7, State: Initialized, Role: FOLLOWER
I20260812 16:52:42.843313 24437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } }
I20260812 16:52:42.843595 24437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:42.843732 24437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:42.843863 24437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:42.846086 24437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } }
I20260812 16:52:42.846617 24437 leader_election.cc:302] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ccc996b6474479f8aa040d507c5bee7; no voters:
I20260812 16:52:42.847227 24437 leader_election.cc:288] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:42.847388 24448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:42.848577 24448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [term 1 LEADER]: Becoming Leader. State: Replica: 9ccc996b6474479f8aa040d507c5bee7, State: Running, Role: LEADER
I20260812 16:52:42.849155 24448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } }
I20260812 16:52:42.849584 24437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:42.852316 24452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ccc996b6474479f8aa040d507c5bee7. Latest consensus state: current_term: 1 leader_uuid: "9ccc996b6474479f8aa040d507c5bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } } }
I20260812 16:52:42.852350 24451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ccc996b6474479f8aa040d507c5bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccc996b6474479f8aa040d507c5bee7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35181 } } }
I20260812 16:52:42.853197 24451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:42.853313 24452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:42.854063 24463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:42.859661 24463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:42.872274 24463 catalog_manager.cc:1347] Generated new cluster ID: d671c47eb34c4d678e1770f22e28c211
I20260812 16:52:42.872409 24463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:42.888402 24463 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:42.889827 24463 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:42.898330 24463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Generated new TSK 0
I20260812 16:52:42.899196 24463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
I20260812 16:52:42.989693 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35181
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:43.346550 24484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:43.346868 24484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:43.346969 24484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:43.354681 24484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:43.354816 24484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:43.354905 24484 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:52:43.354990 24484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:43.365641 24484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:35181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:43.367169 24484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:43.368607 24484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:43.382682 24493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:43.382853 24495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:43.382884 24492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:43.384155 24484 server_base.cc:1034] running on GCE node
I20260812 16:52:43.384931 24484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:43.386044 24484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:43.387331 24484 hybrid_clock.cc:648] HybridClock initialized: now 1786553563387232 us; error 54 us; skew 500 ppm
I20260812 16:52:43.387665 24484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:43 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553563, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:43.391242 24484 init.cc:377] Logged in from keytab as oryx/127.8.195.129@KRBTEST.COM (short username oryx)
I20260812 16:52:43.392990 24484 webserver.cc:460] Webserver started at http://127.8.195.129:33129/ using document root <none> and password file <none>
I20260812 16:52:43.393806 24484 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:43.393949 24484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:43.394311 24484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:43.397576 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "20cfd21a2af6418885901a403f4201e8"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "50e3230bc763669ca5338f7860b5375d"
server_key_iv: "db3228c24620b0e8ad9651e9c496240a"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.398530 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "20cfd21a2af6418885901a403f4201e8"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "50e3230bc763669ca5338f7860b5375d"
server_key_iv: "db3228c24620b0e8ad9651e9c496240a"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.403915 24484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:52:43.407531 24507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:43.408874 24484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:43.409052 24484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "20cfd21a2af6418885901a403f4201e8"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "50e3230bc763669ca5338f7860b5375d"
server_key_iv: "db3228c24620b0e8ad9651e9c496240a"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.409369 24484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:43.450323 24484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:43.455535 24484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:43.455907 24484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:43.459081 24484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:43.459205 24484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:43.459393 24484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:43.459528 24484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 16:52:43 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553563, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:43.488620 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:43.459637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34221 (local address 127.8.195.190:35181)
0812 16:52:43.462762 (+ 3125us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:43.462767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:43.462788 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:52:43.465316 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:43.465326 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:43.465469 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:43.465575 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:43.469229 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.470343 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.473776 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.474195 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.474393 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:43.479684 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:43.479693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:43.479700 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:43.479788 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:43.484120 (+ 4332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:43.485347 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:43.485353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:43.485357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:43.485554 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:43.486325 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:43.486329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:43.486331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:43.486749 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:43.486887 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:43.488154 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0812 16:52:43.488352 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3009,"thread_start_us":2899,"threads_started":1}
I20260812 16:52:43.490963 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:43.460541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.129:34221)
0812 16:52:43.461714 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:43.461794 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:43.464830 (+ 3036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:43.466336 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:43.466353 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:43.467486 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:43.468904 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:43.468948 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.470594 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.470604 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:43.473581 (+ 2977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:43.473598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.474501 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.474511 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:43.474782 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:43.475672 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:43.475719 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:43.479341 (+ 3622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:43.484343 (+ 5002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:43.484356 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:43.484388 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:43.485150 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:43.485712 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:43.485721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:43.485728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:43.486124 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:43.486903 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:43.486917 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:43.487571 (+ 654us) client_negotiation.cc:769] Sending connection context
0812 16:52:43.487988 (+ 417us) client_negotiation.cc:240] Negotiation successful
0812 16:52:43.488769 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":505,"threads_started":1}
I20260812 16:52:43.514205 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:43.498719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:35227 (local address 127.8.195.190:35181)
0812 16:52:43.498977 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:43.498986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:43.499469 (+ 483us) server_negotiation.cc:407] Connection header received
0812 16:52:43.499633 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:43.499641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:43.499760 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:43.499893 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:43.500898 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.502266 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.504207 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.504643 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.504807 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:43.507735 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:43.507744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:43.507751 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:43.507799 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:43.510920 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:43.511966 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:43.511970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:43.511973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:43.512107 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:43.512610 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:43.512614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:43.512616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:43.512957 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:43.513080 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:43.513734 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 16:52:43.513963 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 16:52:43.514803 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:43.498521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.129:35227)
0812 16:52:43.499342 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:43.499371 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:43.499502 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:43.499930 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:43.499933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:43.500434 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:43.500737 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:43.500746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.502470 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.502478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:43.504025 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:43.504040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:43.504854 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:43.504861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:43.505048 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:43.505863 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:43.505895 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:43.507486 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:43.511083 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:43.511091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:43.511098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:43.511804 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:43.512226 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:43.512233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:43.512240 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:43.512468 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:43.513081 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:43.513089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:43.513393 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 16:52:43.513593 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 16:52:43.514121 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":414,"threads_started":1}
I20260812 16:52:43.518523 24484 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:40817
I20260812 16:52:43.518561 24724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:40817 every 8 connection(s)
I20260812 16:52:43.519747 24484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:43.522111 24729 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35181
I20260812 16:52:43.522364 24729 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:43.522619 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 24484
I20260812 16:52:43.522851 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:43.523200 24729 heartbeater.cc:511] Master 127.8.195.190:35181 requested a full tablet report, sending...
I20260812 16:52:43.523627 8974 external_mini_cluster.cc:1428] Setting key 7ac90921ed494cb68f19a5524a9f1d77
I20260812 16:52:43.525668 24358 ts_manager.cc:194] Registered new tserver with Master: 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817)
I20260812 16:52:43.527218 24358 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:34221
WARNING: no policy specified for oryx/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
I20260812 16:52:43.588590 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35181
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:43.927434 24737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:43.927721 24737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:43.927829 24737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:43.935693 24737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:43.935828 24737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:43.935917 24737 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:52:43.935998 24737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:43.946113 24737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:35181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:43.947446 24737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:43.948938 24737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:43.963418 24748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:43.963318 24752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:43.963374 24749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:43.964316 24737 server_base.cc:1034] running on GCE node
I20260812 16:52:43.965075 24737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:43.966266 24737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:43.967586 24737 hybrid_clock.cc:648] HybridClock initialized: now 1786553563967506 us; error 47 us; skew 500 ppm
I20260812 16:52:43.967950 24737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:43 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553563, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:43.971506 24737 init.cc:377] Logged in from keytab as oryx/127.8.195.130@KRBTEST.COM (short username oryx)
I20260812 16:52:43.973133 24737 webserver.cc:460] Webserver started at http://127.8.195.130:35467/ using document root <none> and password file <none>
I20260812 16:52:43.973963 24737 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:43.974108 24737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:43.974462 24737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:43.977689 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "6b09c5ddc94a618693c627d77f4389c5"
server_key_iv: "8c91f4298a262d25ba5f8305506dd5cc"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.978555 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "6b09c5ddc94a618693c627d77f4389c5"
server_key_iv: "8c91f4298a262d25ba5f8305506dd5cc"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.984009 24737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:52:43.987792 24767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:43.989418 24737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:52:43.989609 24737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
format_stamp: "Formatted at 2026-08-12 16:52:43 on dist-test-slave-z8x9"
server_key: "6b09c5ddc94a618693c627d77f4389c5"
server_key_iv: "8c91f4298a262d25ba5f8305506dd5cc"
server_key_version: "encryptionkey@0"
I20260812 16:52:43.989861 24737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:44.006973 24737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:44.011852 24737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:44.012173 24737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:44.015273 24737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:44.015394 24737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:44.015559 24737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:44.015654 24737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553563, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:44.040869 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.015937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:41763 (local address 127.8.195.190:35181)
0812 16:52:44.016554 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:44.016561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:44.017885 (+ 1324us) server_negotiation.cc:407] Connection header received
0812 16:52:44.020296 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:44.020303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:44.020408 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:44.020559 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:44.023286 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.024138 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.026396 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.027074 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.027275 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.033538 (+ 6263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:44.033548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:44.033556 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:44.033614 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:44.036945 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.038218 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.038224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.038226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.038369 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.038894 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.038899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.038901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.039199 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:44.039382 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:44.040209 (+ 827us) server_negotiation.cc:299] Negotiation successful
0812 16:52:44.040461 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":286,"threads_started":1}
I20260812 16:52:44.042336 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.016672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.130:41763)
0812 16:52:44.017687 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:44.017742 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:44.019901 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:44.020988 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:44.021000 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:44.022063 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:44.023046 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.023081 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.024340 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.024346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.026143 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.026164 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.027441 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.027452 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.027861 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.028958 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:44.028983 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:44.033096 (+ 4113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:44.037150 (+ 4054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.037161 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.037230 (+ 69us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.038036 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.038499 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.038504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.038508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.038752 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.039346 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:44.039355 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:44.039772 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 16:52:44.040056 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 16:52:44.040702 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":437,"threads_started":1}
I20260812 16:52:44.070057 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.052875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:49791 (local address 127.8.195.190:35181)
0812 16:52:44.053052 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:44.053057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:44.054218 (+ 1161us) server_negotiation.cc:407] Connection header received
0812 16:52:44.054467 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:44.054473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:44.054566 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:44.054689 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:44.056329 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.057393 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.059756 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.060217 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.060363 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.063457 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:44.063462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:44.063466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:44.063500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:44.067178 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.067874 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.067881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.067884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.068025 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.068422 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.068428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.068431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.068775 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:44.068940 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:44.069440 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 16:52:44.069705 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:52:44.070304 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.053009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.130:49791)
0812 16:52:44.054073 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:44.054098 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:44.054304 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:44.054719 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:44.054728 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:44.055550 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:44.056082 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.056103 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.057667 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.057674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.059525 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.059546 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.060442 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.060450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.060630 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.061460 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:44.061494 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:44.063188 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:44.067319 (+ 4131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.067324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.067328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.067745 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.068145 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.068150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.068154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.068302 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.068902 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:44.068907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:44.069096 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 16:52:44.069332 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 16:52:44.069742 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":680,"threads_started":1}
I20260812 16:52:44.074373 24737 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:39059
I20260812 16:52:44.074401 24974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:39059 every 8 connection(s)
I20260812 16:52:44.075554 24737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:44.077538 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 24737
I20260812 16:52:44.077759 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:44.077744 24975 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35181
I20260812 16:52:44.077970 24975 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:44.078269 8974 external_mini_cluster.cc:1428] Setting key 4123eff7e3604bacb9ec0dfd5569a3ef
I20260812 16:52:44.078681 24975 heartbeater.cc:511] Master 127.8.195.190:35181 requested a full tablet report, sending...
I20260812 16:52:44.080343 24357 ts_manager.cc:194] Registered new tserver with Master: b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:44.081585 24357 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:41763
WARNING: no policy specified for oryx/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
I20260812 16:52:44.150085 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:35181
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:44.499552 24983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:44.499939 24983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:44.500118 24983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:44.507576 24983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:44.507756 24983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:44.507905 24983 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 16:52:44.508045 24983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:44.519114 24983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.195.190:35181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:44.520771 24983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:44.522607 24983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 16:52:44.530653 24729 heartbeater.cc:503] Master 127.8.195.190:35181 was elected leader, sending a full tablet report...
W20260812 16:52:44.537688 24994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:44.537667 24996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:44.537643 25001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:44.539315 24983 server_base.cc:1034] running on GCE node
I20260812 16:52:44.540066 24983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:44.541078 24983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:44.542368 24983 hybrid_clock.cc:648] HybridClock initialized: now 1786553564542298 us; error 43 us; skew 500 ppm
I20260812 16:52:44.542706 24983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553564, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:44.546069 24983 init.cc:377] Logged in from keytab as oryx/127.8.195.131@KRBTEST.COM (short username oryx)
I20260812 16:52:44.547551 24983 webserver.cc:460] Webserver started at http://127.8.195.131:38109/ using document root <none> and password file <none>
I20260812 16:52:44.548295 24983 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:44.548391 24983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:44.548746 24983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:44.552001 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "bdb00002de0d428b9855927ea0210ef3"
format_stamp: "Formatted at 2026-08-12 16:52:44 on dist-test-slave-z8x9"
server_key: "33b7d5a101d84a70e2865fef2d42d5df"
server_key_iv: "e68d9bc81b2296f3cca5a08d65e4d121"
server_key_version: "encryptionkey@0"
I20260812 16:52:44.552889 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "bdb00002de0d428b9855927ea0210ef3"
format_stamp: "Formatted at 2026-08-12 16:52:44 on dist-test-slave-z8x9"
server_key: "33b7d5a101d84a70e2865fef2d42d5df"
server_key_iv: "e68d9bc81b2296f3cca5a08d65e4d121"
server_key_version: "encryptionkey@0"
I20260812 16:52:44.558789 24983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:44.562613 25014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:44.564257 24983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:44.564450 24983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "bdb00002de0d428b9855927ea0210ef3"
format_stamp: "Formatted at 2026-08-12 16:52:44 on dist-test-slave-z8x9"
server_key: "33b7d5a101d84a70e2865fef2d42d5df"
server_key_iv: "e68d9bc81b2296f3cca5a08d65e4d121"
server_key_version: "encryptionkey@0"
I20260812 16:52:44.564706 24983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:44.579617 24983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:44.583832 24983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:44.584115 24983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:44.587020 24983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:44.587129 24983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:44.587312 24983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:44.587376 24983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553564, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:44.613530 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.587508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:46933 (local address 127.8.195.190:35181)
0812 16:52:44.588376 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:44.588380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:44.589221 (+ 841us) server_negotiation.cc:407] Connection header received
0812 16:52:44.591347 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:44.591353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:44.591464 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:44.591587 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:44.594213 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.595386 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.598770 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.599361 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.599524 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.605734 (+ 6210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:44.605739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:44.605742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:44.605780 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:44.608643 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.609911 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.609919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.609925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.610087 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.610740 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.610746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.610749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.611256 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:44.611439 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:44.612591 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0812 16:52:44.612929 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":660,"threads_started":1}
I20260812 16:52:44.615379 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.588220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.131:46933)
0812 16:52:44.589041 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:44.589079 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:44.591028 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:44.591847 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:44.591856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:44.592707 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:44.593842 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.593911 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.595577 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.595588 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.598551 (+ 2963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.598573 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.599688 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.599700 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.600093 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.601012 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:44.601055 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:44.605402 (+ 4347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:44.608815 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.608827 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.608853 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.609715 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.610241 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.610249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.610255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.610571 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.611407 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:44.611415 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:44.612034 (+ 619us) client_negotiation.cc:769] Sending connection context
0812 16:52:44.612405 (+ 371us) client_negotiation.cc:240] Negotiation successful
0812 16:52:44.613198 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":404,"threads_started":1}
I20260812 16:52:44.640096 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.625309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.8.195.131:38067)
0812 16:52:44.626280 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:44.626304 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:44.626483 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:44.626850 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:44.626855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:44.627377 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:44.627678 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.627688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.628706 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.628711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.630057 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:44.630069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.630974 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.630985 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:44.631230 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.632102 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:44.632133 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:44.633726 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:44.637077 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.637083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.637087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.637734 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.638309 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:44.638313 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:44.638315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:44.638440 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:44.639218 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:44.639222 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:44.639410 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 16:52:44.639517 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:52:44.639760 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":696,"threads_started":1}
I20260812 16:52:44.640548 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:44.625366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:38067 (local address 127.8.195.190:35181)
0812 16:52:44.625595 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:44.625603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:44.626404 (+ 801us) server_negotiation.cc:407] Connection header received
0812 16:52:44.626620 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:44.626624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:44.626708 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:44.626826 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:44.627835 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.628551 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.630226 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.630720 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.630874 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:44.633958 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:44.633968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:44.633973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:44.634021 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:44.636933 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.637914 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.637922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.637925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.638157 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:44.638577 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:44.638585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:44.638591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:44.639088 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:44.639287 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:44.639722 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 16:52:44.640047 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:52:44.648463 24983 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:38515
I20260812 16:52:44.648491 25254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:38515 every 8 connection(s)
I20260812 16:52:44.649844 24983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:44.652194 25256 heartbeater.cc:347] Connected to a master server at 127.8.195.190:35181
I20260812 16:52:44.652449 25256 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:44.653126 25256 heartbeater.cc:511] Master 127.8.195.190:35181 requested a full tablet report, sending...
I20260812 16:52:44.654582 24357 ts_manager.cc:194] Registered new tserver with Master: bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515)
I20260812 16:52:44.655265 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 24983
I20260812 16:52:44.655450 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:44.655473 24357 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:46933
I20260812 16:52:44.656102 8974 external_mini_cluster.cc:1428] Setting key 199dff8b2bf2605ac8ac75c50768fff5
I20260812 16:52:44.659305 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:44 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:45.084247 24975 heartbeater.cc:503] Master 127.8.195.190:35181 was elected leader, sending a full tablet report...
I20260812 16:52:45.658399 25256 heartbeater.cc:503] Master 127.8.195.190:35181 was elected leader, sending a full tablet report...
W20260812 16:52:45.812842 25267 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.080s user 0.008s sys 0.012s
W20260812 16:52:45.814189 25035 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:44.727097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.8.195.190:35181)
0812 16:52:44.727344 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:44.727351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:44.727661 (+ 310us) server_negotiation.cc:407] Connection header received
0812 16:52:44.727923 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:44.727927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:44.728000 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:44.728114 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:44.729673 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.730434 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.731712 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:44.732131 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:44.732270 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.813895 (+1081625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39856: BlockingRecv error: recv got EOF from 127.0.0.1:39856 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 16:52:45 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:45.834831 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.819940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.8.195.190:35181)
0812 16:52:45.820130 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:45.820136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:45.820530 (+ 394us) server_negotiation.cc:407] Connection header received
0812 16:52:45.820660 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:45.820664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:45.820741 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:45.820840 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:45.822297 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.823054 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.824400 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.824858 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.824964 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.829103 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:45.829110 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:45.829113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:45.829249 (+ 136us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:45.832302 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:45.832995 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:45.833000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:45.833003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:45.833136 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:45.833567 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:45.833571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:45.833573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:45.833877 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:45.834095 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:45.834362 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 16:52:45.834556 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:52:45.839556 24357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:45.847374 24357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:45.883497 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.862256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.8.195.130:39059)
0812 16:52:45.873302 (+ 11046us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:45.873309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:45.873334 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:52:45.873464 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:45.873471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:45.873712 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:45.873952 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:45.874737 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.876724 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.880399 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.881959 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.882107 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.882356 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:45.883015 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 16:52:45.883178 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10857,"thread_start_us":10689,"threads_started":1}
I20260812 16:52:45.883524 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.861575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.8.195.131:38515)
0812 16:52:45.871422 (+ 9847us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:45.871438 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:45.871484 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:52:45.871628 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:45.871642 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:45.871996 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:45.872293 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:45.873481 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.875720 (+ 2239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.880226 (+ 4506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.881959 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.882450 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.882704 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:45.882942 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 16:52:45.883151 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9605,"thread_start_us":12997,"threads_started":1}
I20260812 16:52:45.883561 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.861902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:39059 (local address 127.0.0.1:32800)
0812 16:52:45.870990 (+ 9088us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:45.871017 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:45.871182 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:45.873957 (+ 2775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:45.873965 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:45.874003 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:45.874515 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.874531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.876924 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.876933 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.880069 (+ 3136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.880093 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.882251 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.882259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.882677 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.882707 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 16:52:45.882841 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 16:52:45.883106 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8897,"mutex_wait_us":148,"thread_start_us":8698,"threads_started":1}
I20260812 16:52:45.884470 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.861416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38515 (local address 127.0.0.1:37766)
0812 16:52:45.869848 (+ 8432us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:45.869912 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:45.870170 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:45.872646 (+ 2476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:45.872656 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:45.872700 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:45.873257 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.873276 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.875962 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.875971 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.879853 (+ 3882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.879883 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.882258 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.882264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.882628 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.882698 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 16:52:45.882833 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 16:52:45.884050 (+ 1217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6981,"mutex_wait_us":1052,"thread_start_us":6788,"threads_started":1}
I20260812 16:52:45.887701 24849 tablet_service.cc:1467] Processing CreateTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 (DEFAULT_TABLE table=test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:45.888443 25125 tablet_service.cc:1467] Processing CreateTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 (DEFAULT_TABLE table=test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:45.889259 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc081f65ed14eb1b58c3e0ea804cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:45.890110 25125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc081f65ed14eb1b58c3e0ea804cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:45.891269 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.869812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.8.195.129:40817)
0812 16:52:45.881688 (+ 11876us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:45.881698 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:45.881729 (+ 31us) server_negotiation.cc:407] Connection header received
0812 16:52:45.881832 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:45.881841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:45.882124 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:45.882329 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:45.883375 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.885690 (+ 2315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.888516 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.890070 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.890231 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.890436 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:45.890772 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 16:52:45.890942 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11704,"thread_start_us":11367,"threads_started":1}
I20260812 16:52:45.891327 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:45.869493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:40817 (local address 127.0.0.1:56988)
0812 16:52:45.880794 (+ 11301us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:45.880815 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:45.880944 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:45.882487 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:45.882491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:45.882520 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:45.883093 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.883109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.885925 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.885932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.888323 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:45.888341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:45.890342 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:45.890350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:45.890550 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:45.890589 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 16:52:45.890712 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 16:52:45.890871 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11147,"mutex_wait_us":3,"thread_start_us":11001,"threads_started":1}
I20260812 16:52:45.895196 24596 tablet_service.cc:1467] Processing CreateTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 (DEFAULT_TABLE table=test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:45.896698 24596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc081f65ed14eb1b58c3e0ea804cc35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:45.903719 25299 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Bootstrap starting.
I20260812 16:52:45.908273 25299 tablet_bootstrap.cc:654] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:45.909344 25301 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Bootstrap starting.
I20260812 16:52:45.909868 25299 log.cc:824] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:45.912506 25304 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Bootstrap starting.
I20260812 16:52:45.913276 25301 tablet_bootstrap.cc:654] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:45.914892 25301 log.cc:824] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:45.915545 25299 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: No bootstrap required, opened a new log
I20260812 16:52:45.915904 25304 tablet_bootstrap.cc:654] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:45.915941 25299 ts_tablet_manager.cc:1397] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 16:52:45.917095 25304 log.cc:824] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:45.918622 25301 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: No bootstrap required, opened a new log
I20260812 16:52:45.918951 25301 ts_tablet_manager.cc:1397] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 16:52:45.919863 25304 tablet_bootstrap.cc:492] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: No bootstrap required, opened a new log
I20260812 16:52:45.920118 25304 ts_tablet_manager.cc:1397] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 16:52:45.928292 25304 raft_consensus.cc:348] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.928326 25299 raft_consensus.cc:348] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.928735 25304 raft_consensus.cc:374] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:45.928855 25304 raft_consensus.cc:729] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Initialized, Role: FOLLOWER
I20260812 16:52:45.928860 25299 raft_consensus.cc:374] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:45.928987 25299 raft_consensus.cc:729] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Initialized, Role: FOLLOWER
I20260812 16:52:45.929570 25304 consensus_queue.cc:260] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.929690 25299 consensus_queue.cc:260] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.931159 25301 raft_consensus.cc:348] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.931645 25301 raft_consensus.cc:374] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:45.931792 25301 raft_consensus.cc:729] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e0d1820cda4a1cb88f01034f6bac96, State: Initialized, Role: FOLLOWER
I20260812 16:52:45.932430 25301 consensus_queue.cc:260] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:45.933367 25299 ts_tablet_manager.cc:1428] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260812 16:52:45.935777 25304 ts_tablet_manager.cc:1428] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Time spent starting tablet: real 0.015s user 0.005s sys 0.011s
I20260812 16:52:45.940250 25301 ts_tablet_manager.cc:1428] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent starting tablet: real 0.021s user 0.018s sys 0.004s
I20260812 16:52:46.004832 25313 raft_consensus.cc:482] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:46.005281 25313 raft_consensus.cc:504] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:46.007261 25313 leader_election.cc:288] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.016680 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.007846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:51489 (local address 127.8.195.130:39059)
0812 16:52:46.008039 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.008046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.008870 (+ 824us) server_negotiation.cc:407] Connection header received
0812 16:52:46.009012 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.009016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.009078 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.009159 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:46.009883 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.011045 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.014149 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.015429 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.015587 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.015665 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.016293 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.016421 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:52:46.016680 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.007797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:46199 (local address 127.8.195.129:40817)
0812 16:52:46.008023 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.008030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.008815 (+ 785us) server_negotiation.cc:407] Connection header received
0812 16:52:46.009407 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.009411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.009470 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.009541 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:46.010185 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.011226 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.014114 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.015417 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.015587 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.015665 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.016272 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.016413 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 16:52:46.016880 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.007551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:39059 (local address 127.8.195.131:51489)
0812 16:52:46.008677 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:46.008699 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:46.008867 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:46.009248 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:46.009252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:46.009274 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:46.009698 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.009713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.011194 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.011201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.013810 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.013825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.015785 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.015791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.016014 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.016120 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 16:52:46.016225 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 16:52:46.016377 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":122,"thread_start_us":759,"threads_started":1}
I20260812 16:52:46.016917 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.007641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:40817 (local address 127.8.195.131:46199)
0812 16:52:46.008680 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:46.008699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:46.008874 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:46.009603 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:46.009609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:46.009641 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:46.009995 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.010003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.011375 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.011386 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.013804 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.013822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.015770 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.015778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.015983 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.016119 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 16:52:46.016225 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 16:52:46.016390 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":122,"spinlock_wait_cycles":39808,"thread_start_us":605,"threads_started":1}
I20260812 16:52:46.018401 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc081f65ed14eb1b58c3e0ea804cc35" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" is_pre_election: true
I20260812 16:52:46.018401 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc081f65ed14eb1b58c3e0ea804cc35" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8" is_pre_election: true
I20260812 16:52:46.019078 24888 raft_consensus.cc:2454] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.019078 24639 raft_consensus.cc:2454] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.019860 25022 leader_election.cc:302] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, b7e0d1820cda4a1cb88f01034f6bac96, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.020350 25313 raft_consensus.cc:2781] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:46.020531 25313 raft_consensus.cc:482] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:46.020653 25313 raft_consensus.cc:3037] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.023036 25313 raft_consensus.cc:504] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
W20260812 16:52:46.023304 24730 tablet.cc:2369] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:52:46.023847 25313 leader_election.cc:288] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.024400 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc081f65ed14eb1b58c3e0ea804cc35" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
I20260812 16:52:46.024399 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc081f65ed14eb1b58c3e0ea804cc35" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8"
I20260812 16:52:46.024626 24639 raft_consensus.cc:3037] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.024622 24888 raft_consensus.cc:3037] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.027935 24639 raft_consensus.cc:2454] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.027935 24888 raft_consensus.cc:2454] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.028425 25023 leader_election.cc:302] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7e0d1820cda4a1cb88f01034f6bac96, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.028811 25313 raft_consensus.cc:2781] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:46.029394 25313 raft_consensus.cc:686] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Becoming Leader. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Running, Role: LEADER
I20260812 16:52:46.029873 25313 consensus_queue.cc:237] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } }
I20260812 16:52:46.034659 24358 catalog_manager.cc:5495] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdb00002de0d428b9855927ea0210ef3 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } health_report { overall_health: UNKNOWN } } }
I20260812 16:52:46.054013 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.048439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.8.195.131:38515)
0812 16:52:46.048684 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.048690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.048712 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:52:46.048879 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.048883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.048963 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.049132 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:46.049564 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.050386 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.051744 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.052140 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.052293 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.052761 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:46.053419 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:46.053578 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.053643 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.053730 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:52:46.062130 24888 raft_consensus.cc:1261] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.062170 24639 raft_consensus.cc:1261] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.063165 25322 consensus_queue.cc:1035] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.063512 25313 consensus_queue.cc:1035] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.082297 25328 mvcc.cc:205] Tried to move back new op lower bound from 7317723406581059584 to 7317723406462464000. Current Snapshot: MvccSnapshot[applied={T|T < 7317723406581059584}]
I20260812 16:52:46.097556 24357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:46.097896 24357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:46.101980 24357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Sending DeleteTablet for 3 replicas of tablet bbc081f65ed14eb1b58c3e0ea804cc35
I20260812 16:52:46.103346 24596 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:56988
I20260812 16:52:46.103489 25125 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 16:52:46.103637 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc081f65ed14eb1b58c3e0ea804cc35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:32800
I20260812 16:52:46.105358 24357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:46.106806 24357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:46.113791 25348 tablet_replica.cc:330] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: stopping tablet replica
I20260812 16:52:46.114308 25348 raft_consensus.cc:2229] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:46.114876 25348 raft_consensus.cc:2258] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.119414 24594 tablet_service.cc:1467] Processing CreateTablet for tablet bf30a4da62394685915d5a1a446e7e2a (DEFAULT_TABLE table=test-table [id=551a37d87b9747be97264b0f45692d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.119765 25124 tablet_service.cc:1467] Processing CreateTablet for tablet bf30a4da62394685915d5a1a446e7e2a (DEFAULT_TABLE table=test-table [id=551a37d87b9747be97264b0f45692d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.120064 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf30a4da62394685915d5a1a446e7e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.120572 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf30a4da62394685915d5a1a446e7e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.120536 24849 tablet_service.cc:1467] Processing CreateTablet for tablet bf30a4da62394685915d5a1a446e7e2a (DEFAULT_TABLE table=test-table [id=551a37d87b9747be97264b0f45692d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.120986 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf30a4da62394685915d5a1a446e7e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.121713 25346 tablet_replica.cc:330] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: stopping tablet replica
I20260812 16:52:46.122172 25346 raft_consensus.cc:2229] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:46.122891 25346 raft_consensus.cc:2258] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.123751 25342 tablet_replica.cc:330] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: stopping tablet replica
I20260812 16:52:46.124138 25342 raft_consensus.cc:2229] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:46.124682 25342 raft_consensus.cc:2258] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.127008 25346 ts_tablet_manager.cc:1905] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.127764 25304 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Bootstrap starting.
I20260812 16:52:46.127990 25299 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Bootstrap starting.
I20260812 16:52:46.129544 25304 tablet_bootstrap.cc:654] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.129737 25299 tablet_bootstrap.cc:654] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.132616 25301 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Bootstrap starting.
I20260812 16:52:46.134554 25301 tablet_bootstrap.cc:654] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.135481 25346 ts_tablet_manager.cc:1918] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:46.135717 25346 log.cc:1196] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/bbc081f65ed14eb1b58c3e0ea804cc35
I20260812 16:52:46.136003 25348 ts_tablet_manager.cc:1905] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.136513 25346 ts_tablet_manager.cc:1939] T bbc081f65ed14eb1b58c3e0ea804cc35 P bdb00002de0d428b9855927ea0210ef3: Deleting consensus metadata
I20260812 16:52:46.137899 25342 ts_tablet_manager.cc:1905] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.138468 24333 catalog_manager.cc:4841] TS bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515): tablet bbc081f65ed14eb1b58c3e0ea804cc35 (table test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]) successfully deleted
I20260812 16:52:46.147017 25304 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: No bootstrap required, opened a new log
I20260812 16:52:46.147307 25304 ts_tablet_manager.cc:1397] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.005s
I20260812 16:52:46.148232 25301 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: No bootstrap required, opened a new log
I20260812 16:52:46.148190 25342 ts_tablet_manager.cc:1918] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:46.148492 25301 ts_tablet_manager.cc:1397] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260812 16:52:46.148411 25342 log.cc:1196] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/bbc081f65ed14eb1b58c3e0ea804cc35
I20260812 16:52:46.148411 25304 raft_consensus.cc:348] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.148766 25304 raft_consensus.cc:374] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.148929 25304 raft_consensus.cc:729] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.149494 25301 raft_consensus.cc:348] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.149523 25304 consensus_queue.cc:260] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.149794 25301 raft_consensus.cc:374] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.149956 25301 raft_consensus.cc:729] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e0d1820cda4a1cb88f01034f6bac96, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.150342 25301 consensus_queue.cc:260] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.151577 25301 ts_tablet_manager.cc:1428] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:46.153879 25342 ts_tablet_manager.cc:1939] T bbc081f65ed14eb1b58c3e0ea804cc35 P 20cfd21a2af6418885901a403f4201e8: Deleting consensus metadata
I20260812 16:52:46.154631 25304 ts_tablet_manager.cc:1428] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260812 16:52:46.157562 24333 catalog_manager.cc:4841] TS 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817): tablet bbc081f65ed14eb1b58c3e0ea804cc35 (table test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]) successfully deleted
I20260812 16:52:46.160938 25299 tablet_bootstrap.cc:492] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: No bootstrap required, opened a new log
I20260812 16:52:46.161150 25299 ts_tablet_manager.cc:1397] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.009s
I20260812 16:52:46.162106 25299 raft_consensus.cc:348] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.162328 25299 raft_consensus.cc:374] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.162307 25348 ts_tablet_manager.cc:1918] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:46.162438 25299 raft_consensus.cc:729] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.162496 25348 log.cc:1196] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/bbc081f65ed14eb1b58c3e0ea804cc35
I20260812 16:52:46.162755 25299 consensus_queue.cc:260] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.163220 25348 ts_tablet_manager.cc:1939] T bbc081f65ed14eb1b58c3e0ea804cc35 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting consensus metadata
I20260812 16:52:46.165055 25299 ts_tablet_manager.cc:1428] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 16:52:46.165230 24335 catalog_manager.cc:4841] TS b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059): tablet bbc081f65ed14eb1b58c3e0ea804cc35 (table test-table [id=b3e7675d2dbf4cb0b3f601f5b2c0ed0f]) successfully deleted
W20260812 16:52:46.329701 24978 tablet.cc:2369] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:52:46.356006 25338 raft_consensus.cc:482] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:46.356465 25338 raft_consensus.cc:504] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.357672 25338 leader_election.cc:288] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.358342 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf30a4da62394685915d5a1a446e7e2a" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8" is_pre_election: true
I20260812 16:52:46.358382 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf30a4da62394685915d5a1a446e7e2a" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" is_pre_election: true
I20260812 16:52:46.358633 24639 raft_consensus.cc:2454] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.358670 24888 raft_consensus.cc:2454] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.359118 25022 leader_election.cc:302] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.359429 25338 raft_consensus.cc:2781] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:46.359571 25338 raft_consensus.cc:482] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:46.359706 25338 raft_consensus.cc:3037] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.362751 25338 raft_consensus.cc:504] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.363462 25338 leader_election.cc:288] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.363925 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf30a4da62394685915d5a1a446e7e2a" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8"
I20260812 16:52:46.364001 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf30a4da62394685915d5a1a446e7e2a" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
I20260812 16:52:46.364146 24639 raft_consensus.cc:3037] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.364248 24888 raft_consensus.cc:3037] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.366478 24639 raft_consensus.cc:2454] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.366961 25022 leader_election.cc:302] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.367242 25338 raft_consensus.cc:2781] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:46.367257 24888 raft_consensus.cc:2454] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.367425 25338 raft_consensus.cc:686] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Becoming Leader. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Running, Role: LEADER
I20260812 16:52:46.367795 25338 consensus_queue.cc:237] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.371004 24357 catalog_manager.cc:5495] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdb00002de0d428b9855927ea0210ef3 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } health_report { overall_health: HEALTHY } } }
W20260812 16:52:46.402717 25257 tablet.cc:2369] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:52:46 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553562, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.190@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:46.473897 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.456275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.8.195.190:35181)
0812 16:52:46.457517 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.457522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.457545 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:46.457605 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.457609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.457676 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.457776 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:46.459376 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.460287 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.462215 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.462681 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.462801 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.467432 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:46.467438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:46.467441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:46.467472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:46.470691 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:46.471669 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:46.471673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:46.471675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:46.471816 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:46.472244 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:46.472249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:46.472251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:46.472528 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:46.472641 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.473271 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.473594 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":1123,"thread_start_us":940,"threads_started":1}
I20260812 16:52:46.473987 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.456240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:35181 (local address 127.0.0.1:39872)
0812 16:52:46.457145 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:46.457192 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:46.457409 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:46.457887 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:46.457892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:46.458903 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:46.459186 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.459203 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.460451 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.460457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.462042 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.462059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.462901 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.462910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.463103 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.463907 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:46.463937 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:46.467222 (+ 3285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:46.470848 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:46.470863 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:46.470872 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:46.471537 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:46.471938 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:46.471943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:46.471947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:46.472128 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:46.472673 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:46.472682 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:46.472889 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 16:52:46.473100 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 16:52:46.473460 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":392,"threads_started":1}
I20260812 16:52:46.480280 24357 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:39872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 16:52:46.481730 24357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:46.489514 24849 tablet_service.cc:1467] Processing CreateTablet for tablet e0fabeb077a446e6a2b8f3e2122c3202 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2d6d4094f7430c9d0315d829093dfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:52:46.489451 24594 tablet_service.cc:1467] Processing CreateTablet for tablet e0fabeb077a446e6a2b8f3e2122c3202 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2d6d4094f7430c9d0315d829093dfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:52:46.490060 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fabeb077a446e6a2b8f3e2122c3202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.490083 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fabeb077a446e6a2b8f3e2122c3202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.493894 25124 tablet_service.cc:1467] Processing CreateTablet for tablet e0fabeb077a446e6a2b8f3e2122c3202 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0e2d6d4094f7430c9d0315d829093dfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 16:52:46.494522 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fabeb077a446e6a2b8f3e2122c3202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.496625 25301 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: Bootstrap starting.
I20260812 16:52:46.498060 25304 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: Bootstrap starting.
I20260812 16:52:46.498481 25301 tablet_bootstrap.cc:654] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.499877 25304 tablet_bootstrap.cc:654] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.505492 25299 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: Bootstrap starting.
I20260812 16:52:46.508117 25299 tablet_bootstrap.cc:654] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.510488 25301 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: No bootstrap required, opened a new log
I20260812 16:52:46.510682 25301 ts_tablet_manager.cc:1397] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260812 16:52:46.511489 25301 raft_consensus.cc:348] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.511792 25301 raft_consensus.cc:374] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.511895 25301 raft_consensus.cc:729] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e0d1820cda4a1cb88f01034f6bac96, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.511979 25304 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: No bootstrap required, opened a new log
I20260812 16:52:46.512169 25304 ts_tablet_manager.cc:1397] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260812 16:52:46.512264 25301 consensus_queue.cc:260] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.512940 25304 raft_consensus.cc:348] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.513314 25304 raft_consensus.cc:374] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.513406 25304 raft_consensus.cc:729] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.513703 25304 consensus_queue.cc:260] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.512879 25315 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.514142 25315 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.514199 25312 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.514609 25312 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.515105 25304 ts_tablet_manager.cc:1428] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:46.520865 25301 ts_tablet_manager.cc:1428] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent starting tablet: real 0.010s user 0.002s sys 0.001s
I20260812 16:52:46.521379 25299 tablet_bootstrap.cc:492] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: No bootstrap required, opened a new log
I20260812 16:52:46.521569 25299 ts_tablet_manager.cc:1397] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.006s
I20260812 16:52:46.522445 25299 raft_consensus.cc:348] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.522722 25299 raft_consensus.cc:374] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.522823 25299 raft_consensus.cc:729] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.523121 25299 consensus_queue.cc:260] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.523613 25338 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.523939 25338 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.525599 25299 ts_tablet_manager.cc:1428] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 16:52:46.645671 25338 raft_consensus.cc:482] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:46.645931 25338 raft_consensus.cc:504] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.646626 25338 leader_election.cc:288] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.647280 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fabeb077a446e6a2b8f3e2122c3202" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" is_pre_election: true
I20260812 16:52:46.647310 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fabeb077a446e6a2b8f3e2122c3202" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8" is_pre_election: true
I20260812 16:52:46.647533 24888 raft_consensus.cc:2454] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.647621 24639 raft_consensus.cc:2454] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:46.647959 25023 leader_election.cc:302] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7e0d1820cda4a1cb88f01034f6bac96, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.648368 25338 raft_consensus.cc:2781] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:46.648469 25338 raft_consensus.cc:482] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:46.648595 25338 raft_consensus.cc:3037] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.651095 25338 raft_consensus.cc:504] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.651705 25338 leader_election.cc:288] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:46.652153 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fabeb077a446e6a2b8f3e2122c3202" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8"
I20260812 16:52:46.652258 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fabeb077a446e6a2b8f3e2122c3202" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
I20260812 16:52:46.652427 24639 raft_consensus.cc:3037] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.652494 24888 raft_consensus.cc:3037] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:46.655398 24639 raft_consensus.cc:2454] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.655424 24888 raft_consensus.cc:2454] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:46.655887 25022 leader_election.cc:302] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:46.656173 25338 raft_consensus.cc:2781] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:46.656551 25338 raft_consensus.cc:686] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Becoming Leader. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Running, Role: LEADER
I20260812 16:52:46.656801 25338 consensus_queue.cc:237] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.658452 25313 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: TxnStatusTablet state changed. Reason: New leader bdb00002de0d428b9855927ea0210ef3. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.658733 25313 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:52:46.659349 25383 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 16:52:46.659586 25383 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 16:52:46.660960 24354 catalog_manager.cc:5495] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdb00002de0d428b9855927ea0210ef3 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260812 16:52:46.686486 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.679780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.8.195.131:38515)
0812 16:52:46.680924 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.680930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.680949 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:52:46.681016 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.681021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.681101 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.681251 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:46.681726 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.682575 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.684512 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.684976 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.685071 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.685724 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:46.685829 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:46.685904 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.686098 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.686167 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":820,"threads_started":1}
I20260812 16:52:46.686570 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.679818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38515 (local address 127.0.0.1:37774)
0812 16:52:46.680207 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:46.680238 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:46.680366 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:46.681319 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:46.681324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:46.681339 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:52:46.681550 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.681562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.682735 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.682743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.684355 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.684371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.685135 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.685141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.685441 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.685523 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:52:46.685942 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:52:46.685977 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:52:46.686071 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 16:52:46.686237 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260812 16:52:46.690748 24639 raft_consensus.cc:1261] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.690831 24888 raft_consensus.cc:1261] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.691478 25313 consensus_queue.cc:1035] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.691768 25338 consensus_queue.cc:1035] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.698087 25312 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: TxnStatusTablet state changed. Reason: New leader bdb00002de0d428b9855927ea0210ef3. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.698392 25312 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.701269 25315 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: TxnStatusTablet state changed. Reason: New leader bdb00002de0d428b9855927ea0210ef3. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.701560 25315 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.702342 25391 mvcc.cc:205] Tried to move back new op lower bound from 7317723409158615040 to 7317723409029005312. Current Snapshot: MvccSnapshot[applied={T|T < 7317723409158615040}]
I20260812 16:52:46.709790 25313 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.710186 25313 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:52:46.710628 25312 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.710891 25312 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P 20cfd21a2af6418885901a403f4201e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.711777 25313 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.712078 25313 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P bdb00002de0d428b9855927ea0210ef3: This TxnStatusTablet replica's current role is: LEADER
I20260812 16:52:46.716645 25315 tablet_replica.cc:439] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } } }
I20260812 16:52:46.716995 25315 tablet_replica.cc:442] T e0fabeb077a446e6a2b8f3e2122c3202 P b7e0d1820cda4a1cb88f01034f6bac96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 16:52:46.734552 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.727743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:34215 (local address 127.8.195.131:38515)
0812 16:52:46.727942 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.727949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.728555 (+ 606us) server_negotiation.cc:407] Connection header received
0812 16:52:46.728714 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.728721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.728817 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.728932 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:46.729509 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.730820 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.732325 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.733028 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.733222 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.733707 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:46.733850 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:46.733965 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.734125 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.734219 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:52:46.734552 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.727747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38515 (local address 127.8.195.131:34215)
0812 16:52:46.728452 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:46.728469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:46.728588 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:46.728950 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:46.728953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:46.728963 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:52:46.729313 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.729322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.730960 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.730963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.732177 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:46.732188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.733307 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.733313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:46.733459 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.733518 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:52:46.733973 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:52:46.733983 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:52:46.734057 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 16:52:46.734185 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":364,"threads_started":1}
I20260812 16:52:46.744596 24888 raft_consensus.cc:1261] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.744596 24639 raft_consensus.cc:1261] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:46.745277 25313 consensus_queue.cc:1035] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.745641 25338 consensus_queue.cc:1035] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:46.811269 24357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:46.811529 24357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:46.814688 24357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Sending DeleteTablet for 3 replicas of tablet bf30a4da62394685915d5a1a446e7e2a
I20260812 16:52:46.815536 24594 tablet_service.cc:1514] Processing DeleteTablet for tablet bf30a4da62394685915d5a1a446e7e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:56988
I20260812 16:52:46.815807 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet bf30a4da62394685915d5a1a446e7e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:32800
I20260812 16:52:46.816975 25124 tablet_service.cc:1514] Processing DeleteTablet for tablet bf30a4da62394685915d5a1a446e7e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:46 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 16:52:46.838081 25427 tablet_replica.cc:330] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: stopping tablet replica
I20260812 16:52:46.838399 25427 raft_consensus.cc:2229] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:46.838716 25423 tablet_replica.cc:330] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: stopping tablet replica
I20260812 16:52:46.838953 25427 raft_consensus.cc:2258] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.838934 25423 raft_consensus.cc:2229] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:46.839215 25423 raft_consensus.cc:2258] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.839570 25425 tablet_replica.cc:330] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: stopping tablet replica
I20260812 16:52:46.839808 25425 raft_consensus.cc:2229] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:46.840078 25425 raft_consensus.cc:2258] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:46.841487 25425 ts_tablet_manager.cc:1905] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.844075 25427 ts_tablet_manager.cc:1905] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.845031 25423 ts_tablet_manager.cc:1905] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:46.848912 25425 ts_tablet_manager.cc:1918] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:46.849068 25425 log.cc:1196] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/bf30a4da62394685915d5a1a446e7e2a
I20260812 16:52:46.849653 25425 ts_tablet_manager.cc:1939] T bf30a4da62394685915d5a1a446e7e2a P b7e0d1820cda4a1cb88f01034f6bac96: Deleting consensus metadata
I20260812 16:52:46.850976 24335 catalog_manager.cc:4841] TS b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059): tablet bf30a4da62394685915d5a1a446e7e2a (table test-table [id=551a37d87b9747be97264b0f45692d94]) successfully deleted
I20260812 16:52:46.852411 25427 ts_tablet_manager.cc:1918] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:46.852560 25427 log.cc:1196] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/bf30a4da62394685915d5a1a446e7e2a
I20260812 16:52:46.852751 25423 ts_tablet_manager.cc:1918] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:46.852924 25423 log.cc:1196] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/bf30a4da62394685915d5a1a446e7e2a
I20260812 16:52:46.853098 25427 ts_tablet_manager.cc:1939] T bf30a4da62394685915d5a1a446e7e2a P bdb00002de0d428b9855927ea0210ef3: Deleting consensus metadata
I20260812 16:52:46.853468 25423 ts_tablet_manager.cc:1939] T bf30a4da62394685915d5a1a446e7e2a P 20cfd21a2af6418885901a403f4201e8: Deleting consensus metadata
I20260812 16:52:46.854445 24333 catalog_manager.cc:4841] TS bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515): tablet bf30a4da62394685915d5a1a446e7e2a (table test-table [id=551a37d87b9747be97264b0f45692d94]) successfully deleted
I20260812 16:52:46.854759 24333 catalog_manager.cc:4841] TS 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817): tablet bf30a4da62394685915d5a1a446e7e2a (table test-table [id=551a37d87b9747be97264b0f45692d94]) successfully deleted
Aug 12 16:52:46 dist-test-slave-z8x9 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 16:52:46 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:46 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:46.911788 25369 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:46.899019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.8.195.190:35181)
0812 16:52:46.899284 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.899292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.899747 (+ 455us) server_negotiation.cc:407] Connection header received
0812 16:52:46.899929 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.899936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.900040 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.900186 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:46.901761 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.903050 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.904373 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.904826 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.904975 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.911494 (+ 6519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39880: BlockingRecv error: recv got EOF from 127.0.0.1:39880 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 16:52:46 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM
I20260812 16:52:46.932125 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:46.915502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.8.195.190:35181)
0812 16:52:46.915686 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:46.915691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:46.916167 (+ 476us) server_negotiation.cc:407] Connection header received
0812 16:52:46.916404 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:46.916415 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:46.916485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:46.916602 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:46.918262 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.919174 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.920538 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:46.921021 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:46.921172 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:46.925721 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:46.925727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:46.925730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:46.925770 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:46.929001 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:46.929863 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:46.929872 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:46.929881 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:46.930059 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:46.930501 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:46.930506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:46.930508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:46.930800 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:46.931112 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:46.931446 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 16:52:46.931748 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:52:46.936092 24357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:52:46.936793 24357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:46.944334 24594 tablet_service.cc:1467] Processing CreateTablet for tablet 676cce6e9f8a412a9821883ad374576f (DEFAULT_TABLE table=test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.944620 24849 tablet_service.cc:1467] Processing CreateTablet for tablet 676cce6e9f8a412a9821883ad374576f (DEFAULT_TABLE table=test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.944916 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676cce6e9f8a412a9821883ad374576f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.945115 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676cce6e9f8a412a9821883ad374576f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.950160 25124 tablet_service.cc:1467] Processing CreateTablet for tablet 676cce6e9f8a412a9821883ad374576f (DEFAULT_TABLE table=test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:46.950683 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676cce6e9f8a412a9821883ad374576f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:46.951373 25301 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Bootstrap starting.
I20260812 16:52:46.952217 25304 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Bootstrap starting.
I20260812 16:52:46.953236 25301 tablet_bootstrap.cc:654] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.954039 25304 tablet_bootstrap.cc:654] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.959229 25299 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Bootstrap starting.
I20260812 16:52:46.961011 25299 tablet_bootstrap.cc:654] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:46.961002 25304 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: No bootstrap required, opened a new log
I20260812 16:52:46.961213 25304 ts_tablet_manager.cc:1397] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20260812 16:52:46.962162 25304 raft_consensus.cc:348] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.962473 25304 raft_consensus.cc:374] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.962582 25304 raft_consensus.cc:729] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.962891 25304 consensus_queue.cc:260] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.964329 25299 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: No bootstrap required, opened a new log
I20260812 16:52:46.964322 25301 tablet_bootstrap.cc:492] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: No bootstrap required, opened a new log
I20260812 16:52:46.964506 25299 ts_tablet_manager.cc:1397] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 16:52:46.964550 25301 ts_tablet_manager.cc:1397] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260812 16:52:46.965427 25299 raft_consensus.cc:348] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.965775 25299 raft_consensus.cc:374] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.965901 25299 raft_consensus.cc:729] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.965714 25301 raft_consensus.cc:348] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.966287 25301 raft_consensus.cc:374] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:46.966236 25299 consensus_queue.cc:260] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.966431 25301 raft_consensus.cc:729] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e0d1820cda4a1cb88f01034f6bac96, State: Initialized, Role: FOLLOWER
I20260812 16:52:46.966935 25299 ts_tablet_manager.cc:1428] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 16:52:46.966811 25301 consensus_queue.cc:260] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:46.967644 25301 ts_tablet_manager.cc:1428] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:46.968832 25304 ts_tablet_manager.cc:1428] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260812 16:52:47.214833 25416 raft_consensus.cc:482] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:47.215093 25416 raft_consensus.cc:504] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.215848 25416 leader_election.cc:288] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:47.216537 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676cce6e9f8a412a9821883ad374576f" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8" is_pre_election: true
I20260812 16:52:47.216539 24886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676cce6e9f8a412a9821883ad374576f" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" is_pre_election: true
I20260812 16:52:47.216805 24886 raft_consensus.cc:2454] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:47.216804 24639 raft_consensus.cc:2454] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 0.
I20260812 16:52:47.217374 25023 leader_election.cc:302] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7e0d1820cda4a1cb88f01034f6bac96, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:47.217760 25416 raft_consensus.cc:2781] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:47.217911 25416 raft_consensus.cc:482] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:47.218015 25416 raft_consensus.cc:3037] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.220184 25416 raft_consensus.cc:504] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.220804 25416 leader_election.cc:288] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817), b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059)
I20260812 16:52:47.221410 24639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676cce6e9f8a412a9821883ad374576f" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfd21a2af6418885901a403f4201e8"
I20260812 16:52:47.221441 24886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676cce6e9f8a412a9821883ad374576f" candidate_uuid: "bdb00002de0d428b9855927ea0210ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
I20260812 16:52:47.221643 24639 raft_consensus.cc:3037] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.221704 24886 raft_consensus.cc:3037] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.224496 24886 raft_consensus.cc:2454] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:47.224673 24639 raft_consensus.cc:2454] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb00002de0d428b9855927ea0210ef3 in term 1.
I20260812 16:52:47.225085 25023 leader_election.cc:302] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7e0d1820cda4a1cb88f01034f6bac96, bdb00002de0d428b9855927ea0210ef3; no voters:
I20260812 16:52:47.225443 25416 raft_consensus.cc:2781] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:47.225630 25416 raft_consensus.cc:686] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Becoming Leader. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Running, Role: LEADER
I20260812 16:52:47.225970 25416 consensus_queue.cc:237] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.228838 24354 catalog_manager.cc:5495] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdb00002de0d428b9855927ea0210ef3 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "bdb00002de0d428b9855927ea0210ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } health_report { overall_health: HEALTHY } } }
I20260812 16:52:47.271529 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.266066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.8.195.131:38515)
0812 16:52:47.266659 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.266664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.266687 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:52:47.266805 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.266813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.266892 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.267003 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:47.267405 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.268148 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.269761 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.270169 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.270319 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.270767 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.270874 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.270976 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.271138 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.271232 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":368,"threads_started":1}
I20260812 16:52:47.274487 24639 raft_consensus.cc:1261] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:47.274488 24886 raft_consensus.cc:1261] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Refusing update from remote peer bdb00002de0d428b9855927ea0210ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:47.275205 25338 consensus_queue.cc:1035] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:47.275489 25416 consensus_queue.cc:1035] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:47.282394 25392 mvcc.cc:205] Tried to move back new op lower bound from 7317723411550040064 to 7317723411360079872. Current Snapshot: MvccSnapshot[applied={T|T < 7317723411550040064}]
I20260812 16:52:47.286825 24358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:47.287070 24358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:47.289825 24358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Sending DeleteTablet for 3 replicas of tablet 676cce6e9f8a412a9821883ad374576f
I20260812 16:52:47.290558 24594 tablet_service.cc:1514] Processing DeleteTablet for tablet 676cce6e9f8a412a9821883ad374576f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:56988
I20260812 16:52:47.290786 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet 676cce6e9f8a412a9821883ad374576f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:32800
I20260812 16:52:47.290905 25423 tablet_replica.cc:330] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: stopping tablet replica
I20260812 16:52:47.291070 25425 tablet_replica.cc:330] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: stopping tablet replica
I20260812 16:52:47.291098 25423 raft_consensus.cc:2229] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:47.291240 25425 raft_consensus.cc:2229] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:47.291266 25423 raft_consensus.cc:2258] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.291402 25425 raft_consensus.cc:2258] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.292270 25423 ts_tablet_manager.cc:1905] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.292465 25425 ts_tablet_manager.cc:1905] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.292616 25124 tablet_service.cc:1514] Processing DeleteTablet for tablet 676cce6e9f8a412a9821883ad374576f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 16:52:47.292549 24358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 16:52:47.292881 25427 tablet_replica.cc:330] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: stopping tablet replica
I20260812 16:52:47.293069 25427 raft_consensus.cc:2229] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 1 LEADER]: Raft consensus shutting down.
W20260812 16:52:47.293244 24358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:52:47.293478 25427 raft_consensus.cc:2258] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.294806 25427 ts_tablet_manager.cc:1905] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.299856 25425 ts_tablet_manager.cc:1918] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:47.300138 25425 log.cc:1196] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/676cce6e9f8a412a9821883ad374576f
I20260812 16:52:47.300276 24594 tablet_service.cc:1467] Processing CreateTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 (DEFAULT_TABLE table=test-table [id=34386d455d874732b2b0a3c6e356255d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:47.300762 24594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c3fad98e1444a3adb1553da5ed2536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:47.300685 25124 tablet_service.cc:1467] Processing CreateTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 (DEFAULT_TABLE table=test-table [id=34386d455d874732b2b0a3c6e356255d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:47.301082 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c3fad98e1444a3adb1553da5ed2536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:47.300848 24849 tablet_service.cc:1467] Processing CreateTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 (DEFAULT_TABLE table=test-table [id=34386d455d874732b2b0a3c6e356255d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:52:47.301437 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85c3fad98e1444a3adb1553da5ed2536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:47.302304 25423 ts_tablet_manager.cc:1918] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:47.302462 25423 log.cc:1196] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/676cce6e9f8a412a9821883ad374576f
I20260812 16:52:47.302973 25423 ts_tablet_manager.cc:1939] T 676cce6e9f8a412a9821883ad374576f P 20cfd21a2af6418885901a403f4201e8: Deleting consensus metadata
I20260812 16:52:47.305503 24333 catalog_manager.cc:4841] TS 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817): tablet 676cce6e9f8a412a9821883ad374576f (table test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]) successfully deleted
I20260812 16:52:47.307971 25427 ts_tablet_manager.cc:1918] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:52:47.308141 25299 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Bootstrap starting.
I20260812 16:52:47.308164 25427 log.cc:1196] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/676cce6e9f8a412a9821883ad374576f
I20260812 16:52:47.308750 25301 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Bootstrap starting.
I20260812 16:52:47.309856 25299 tablet_bootstrap.cc:654] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:47.310240 25425 ts_tablet_manager.cc:1939] T 676cce6e9f8a412a9821883ad374576f P b7e0d1820cda4a1cb88f01034f6bac96: Deleting consensus metadata
I20260812 16:52:47.310468 25301 tablet_bootstrap.cc:654] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:47.311411 24335 catalog_manager.cc:4841] TS b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059): tablet 676cce6e9f8a412a9821883ad374576f (table test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]) successfully deleted
I20260812 16:52:47.311534 25304 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Bootstrap starting.
I20260812 16:52:47.312430 25299 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: No bootstrap required, opened a new log
I20260812 16:52:47.312608 25299 ts_tablet_manager.cc:1397] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 16:52:47.312871 25304 tablet_bootstrap.cc:654] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Neither blocks nor log segments found. Creating new log.
I20260812 16:52:47.313513 25299 raft_consensus.cc:348] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.313755 25299 raft_consensus.cc:374] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:47.313851 25299 raft_consensus.cc:729] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb00002de0d428b9855927ea0210ef3, State: Initialized, Role: FOLLOWER
I20260812 16:52:47.314106 25299 consensus_queue.cc:260] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.314770 25299 ts_tablet_manager.cc:1428] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 16:52:47.315080 25304 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: No bootstrap required, opened a new log
I20260812 16:52:47.315271 25304 ts_tablet_manager.cc:1397] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 16:52:47.315888 25304 raft_consensus.cc:348] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.316102 25304 raft_consensus.cc:374] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:47.316183 25304 raft_consensus.cc:729] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Initialized, Role: FOLLOWER
I20260812 16:52:47.316462 25304 consensus_queue.cc:260] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.317057 25304 ts_tablet_manager.cc:1428] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 16:52:47.319751 25427 ts_tablet_manager.cc:1939] T 676cce6e9f8a412a9821883ad374576f P bdb00002de0d428b9855927ea0210ef3: Deleting consensus metadata
I20260812 16:52:47.320953 24333 catalog_manager.cc:4841] TS bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515): tablet 676cce6e9f8a412a9821883ad374576f (table test-table [id=a9b16d7ef2d04f75b2bb1122caee2ec0]) successfully deleted
I20260812 16:52:47.327313 25301 tablet_bootstrap.cc:492] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: No bootstrap required, opened a new log
I20260812 16:52:47.327512 25301 ts_tablet_manager.cc:1397] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.007s
I20260812 16:52:47.328166 25301 raft_consensus.cc:348] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.328399 25301 raft_consensus.cc:374] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:47.328480 25301 raft_consensus.cc:729] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e0d1820cda4a1cb88f01034f6bac96, State: Initialized, Role: FOLLOWER
I20260812 16:52:47.328716 25301 consensus_queue.cc:260] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.329160 25301 ts_tablet_manager.cc:1428] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 16:52:47.540418 25312 raft_consensus.cc:482] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:52:47.540613 25312 raft_consensus.cc:504] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.542362 25312 leader_election.cc:288] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059), bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515)
I20260812 16:52:47.549687 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.542839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:57529 (local address 127.8.195.131:38515)
0812 16:52:47.543036 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.543045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.543540 (+ 495us) server_negotiation.cc:407] Connection header received
0812 16:52:47.543880 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.543885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.543949 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.544053 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:47.544631 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.545436 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.547807 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.548607 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.548823 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.548898 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.549233 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.549372 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 16:52:47.550169 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.542626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38515 (local address 127.8.195.129:57529)
0812 16:52:47.543413 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:47.543438 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:47.543680 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:47.544080 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:47.544085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:47.544140 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:47.544438 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.544447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.545604 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.545609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.547625 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.547638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.548843 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.548850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.549031 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.549056 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 16:52:47.549168 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:52:47.549357 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":2,"spinlock_wait_cycles":15104,"thread_start_us":999,"threads_started":1}
I20260812 16:52:47.550375 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.542626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:39059 (local address 127.8.195.129:45133)
0812 16:52:47.543957 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:47.543971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:47.544150 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:47.544500 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:47.544505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:47.544529 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:52:47.544936 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.544946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.546214 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.546221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.548096 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.548110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.549380 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.549387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.549629 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.549654 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 16:52:47.549761 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:52:47.549904 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":125,"spinlock_wait_cycles":13824,"thread_start_us":347,"threads_started":1}
I20260812 16:52:47.550500 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.542906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:45133 (local address 127.8.195.130:39059)
0812 16:52:47.543656 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.543662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.544078 (+ 416us) server_negotiation.cc:407] Connection header received
0812 16:52:47.544305 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.544309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.544371 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.544480 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:52:47.545132 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.546075 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.548261 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.549113 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.549380 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.549457 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.549804 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.549946 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":461,"threads_started":1}
I20260812 16:52:47.551179 24886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c3fad98e1444a3adb1553da5ed2536" candidate_uuid: "20cfd21a2af6418885901a403f4201e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" is_pre_election: true
I20260812 16:52:47.551182 25167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c3fad98e1444a3adb1553da5ed2536" candidate_uuid: "20cfd21a2af6418885901a403f4201e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb00002de0d428b9855927ea0210ef3" is_pre_election: true
I20260812 16:52:47.551425 24886 raft_consensus.cc:2454] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20cfd21a2af6418885901a403f4201e8 in term 0.
I20260812 16:52:47.551612 25167 raft_consensus.cc:2454] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20cfd21a2af6418885901a403f4201e8 in term 0.
I20260812 16:52:47.551957 24513 leader_election.cc:302] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, b7e0d1820cda4a1cb88f01034f6bac96; no voters:
I20260812 16:52:47.552433 25312 raft_consensus.cc:2781] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:52:47.552575 25312 raft_consensus.cc:482] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:52:47.552706 25312 raft_consensus.cc:3037] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.555013 25312 raft_consensus.cc:504] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.555663 25312 leader_election.cc:288] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [CANDIDATE]: Term 1 election: Requested vote from peers b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059), bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515)
I20260812 16:52:47.556241 24886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c3fad98e1444a3adb1553da5ed2536" candidate_uuid: "20cfd21a2af6418885901a403f4201e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7e0d1820cda4a1cb88f01034f6bac96"
I20260812 16:52:47.556247 25167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85c3fad98e1444a3adb1553da5ed2536" candidate_uuid: "20cfd21a2af6418885901a403f4201e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb00002de0d428b9855927ea0210ef3"
I20260812 16:52:47.556506 24886 raft_consensus.cc:3037] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.556543 25167 raft_consensus.cc:3037] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:47.558806 24886 raft_consensus.cc:2454] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20cfd21a2af6418885901a403f4201e8 in term 1.
I20260812 16:52:47.558907 25167 raft_consensus.cc:2454] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20cfd21a2af6418885901a403f4201e8 in term 1.
I20260812 16:52:47.559352 24513 leader_election.cc:302] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfd21a2af6418885901a403f4201e8, b7e0d1820cda4a1cb88f01034f6bac96; no voters:
I20260812 16:52:47.559711 25312 raft_consensus.cc:2781] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:47.560176 25312 raft_consensus.cc:686] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 1 LEADER]: Becoming Leader. State: Replica: 20cfd21a2af6418885901a403f4201e8, State: Running, Role: LEADER
I20260812 16:52:47.560599 25312 consensus_queue.cc:237] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } }
I20260812 16:52:47.564462 24354 catalog_manager.cc:5495] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20cfd21a2af6418885901a403f4201e8 (127.8.195.129). New cstate: current_term: 1 leader_uuid: "20cfd21a2af6418885901a403f4201e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfd21a2af6418885901a403f4201e8" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 } health_report { overall_health: UNKNOWN } } }
I20260812 16:52:47.624671 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.618576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.8.195.129:40817)
0812 16:52:47.619312 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.619317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.619337 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:52:47.619424 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.619428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.619524 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.619671 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:47.620073 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.620881 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.622279 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.622726 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.622843 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.623312 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.623972 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.624095 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.624182 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.624249 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":470,"threads_started":1}
I20260812 16:52:47.640942 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.635276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:47221 (local address 127.8.195.131:38515)
0812 16:52:47.635494 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.635504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.636127 (+ 623us) server_negotiation.cc:407] Connection header received
0812 16:52:47.636320 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.636324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.636411 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.636495 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:47.636994 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.637957 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.639327 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.639733 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.639832 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.640297 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.640427 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.640503 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.640639 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.640717 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 16:52:47.641049 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.635346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38515 (local address 127.8.195.129:47221)
0812 16:52:47.636024 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:47.636040 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:47.636164 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:47.636527 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:47.636530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:47.636540 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:52:47.636816 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.636823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.638095 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.638099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.639181 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.639194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.639909 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.639915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.640044 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.640097 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.640521 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.640530 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 16:52:47.640596 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 16:52:47.640710 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":360,"threads_started":1}
I20260812 16:52:47.649852 24886 raft_consensus.cc:1261] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Refusing update from remote peer 20cfd21a2af6418885901a403f4201e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:47.650182 25167 raft_consensus.cc:1261] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Refusing update from remote peer 20cfd21a2af6418885901a403f4201e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:52:47.650540 25478 consensus_queue.cc:1035] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7e0d1820cda4a1cb88f01034f6bac96" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:47.650962 25312 consensus_queue.cc:1035] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdb00002de0d428b9855927ea0210ef3" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:52:47.685343 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.678626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:54357 (local address 127.8.195.129:40817)
0812 16:52:47.678792 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.678796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.678924 (+ 128us) server_negotiation.cc:407] Connection header received
0812 16:52:47.679054 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.679059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.679139 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.679233 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:52:47.679816 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.680908 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.683075 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.683705 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.683889 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.684420 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.684567 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.684660 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.684890 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.684978 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 16:52:47.685652 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.678216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:40817 (local address 127.8.195.131:54357)
0812 16:52:47.678827 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:47.678844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:47.678959 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:47.679278 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:47.679283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:47.679297 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 16:52:47.679628 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.679645 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.681075 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.681081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.682880 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:47.682895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.683919 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.683925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:47.684150 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.684224 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 16:52:47.684712 (+ 488us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 16:52:47.684724 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:52:47.684853 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:52:47.685009 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":406,"threads_started":1}
I20260812 16:52:47.726013 24358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:47.726308 24358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:52:47.729657 24358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ccc996b6474479f8aa040d507c5bee7: Sending DeleteTablet for 3 replicas of tablet 85c3fad98e1444a3adb1553da5ed2536
I20260812 16:52:47.730693 24594 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:56988
I20260812 16:52:47.730814 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:32800
I20260812 16:52:47.731045 25423 tablet_replica.cc:330] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: stopping tablet replica
I20260812 16:52:47.731177 25425 tablet_replica.cc:330] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: stopping tablet replica
I20260812 16:52:47.731240 25423 raft_consensus.cc:2229] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:52:47.731366 25425 raft_consensus.cc:2229] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:47.731587 25425 raft_consensus.cc:2258] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.731765 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 24484
I20260812 16:52:47.731827 25423 raft_consensus.cc:2258] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.732857 25425 ts_tablet_manager.cc:1905] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.734490 25423 ts_tablet_manager.cc:1905] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.737727 25124 tablet_service.cc:1514] Processing DeleteTablet for tablet 85c3fad98e1444a3adb1553da5ed2536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:52:47 UTC) from {username='oryx', principal='oryx/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 16:52:47.738155 25427 tablet_replica.cc:330] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: stopping tablet replica
I20260812 16:52:47.738390 25427 raft_consensus.cc:2229] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:52:47.738639 25427 raft_consensus.cc:2258] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:52:47.739851 25427 ts_tablet_manager.cc:1905] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:52:47.739995 25425 ts_tablet_manager.cc:1918] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:47.740146 25425 log.cc:1196] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/85c3fad98e1444a3adb1553da5ed2536
I20260812 16:52:47.740638 25425 ts_tablet_manager.cc:1939] T 85c3fad98e1444a3adb1553da5ed2536 P b7e0d1820cda4a1cb88f01034f6bac96: Deleting consensus metadata
I20260812 16:52:47.741984 24335 catalog_manager.cc:4841] TS b7e0d1820cda4a1cb88f01034f6bac96 (127.8.195.130:39059): tablet 85c3fad98e1444a3adb1553da5ed2536 (table test-table [id=34386d455d874732b2b0a3c6e356255d]) successfully deleted
I20260812 16:52:47.744273 25423 ts_tablet_manager.cc:1918] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:47.744477 25423 log.cc:1196] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/85c3fad98e1444a3adb1553da5ed2536
I20260812 16:52:47.745137 25423 ts_tablet_manager.cc:1939] T 85c3fad98e1444a3adb1553da5ed2536 P 20cfd21a2af6418885901a403f4201e8: Deleting consensus metadata
I20260812 16:52:47.746053 25427 ts_tablet_manager.cc:1918] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 16:52:47.746227 25427 log.cc:1196] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/85c3fad98e1444a3adb1553da5ed2536
I20260812 16:52:47.746714 25427 ts_tablet_manager.cc:1939] T 85c3fad98e1444a3adb1553da5ed2536 P bdb00002de0d428b9855927ea0210ef3: Deleting consensus metadata
I20260812 16:52:47.746855 24333 catalog_manager.cc:4841] TS 20cfd21a2af6418885901a403f4201e8 (127.8.195.129:40817): tablet 85c3fad98e1444a3adb1553da5ed2536 (table test-table [id=34386d455d874732b2b0a3c6e356255d]) successfully deleted
I20260812 16:52:47.747889 24333 catalog_manager.cc:4841] TS bdb00002de0d428b9855927ea0210ef3 (127.8.195.131:38515): tablet 85c3fad98e1444a3adb1553da5ed2536 (table test-table [id=34386d455d874732b2b0a3c6e356255d]) successfully deleted
Aug 12 16:52:47 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.129@KRBTEST.COM
I20260812 16:52:47.767944 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.740328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.8.195.129:40817)
0812 16:52:47.740580 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.740590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.747106 (+ 6516us) server_negotiation.cc:407] Connection header received
0812 16:52:47.747418 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.747429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.747530 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.747649 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:47.749161 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.750110 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.751567 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.752025 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.752173 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.760941 (+ 8768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:47.760957 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:47.760978 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:47.761028 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:47.764409 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:47.765391 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:47.765400 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:47.765402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:47.765567 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:47.766068 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:47.766073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:47.766075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:47.766469 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:47.766859 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.767220 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.767495 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:52:47.768899 24716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:47.885752 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 24484
I20260812 16:52:47.918817 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 24737
Aug 12 16:52:47 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.130@KRBTEST.COM
I20260812 16:52:47.934690 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:47.919690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.8.195.130:39059)
0812 16:52:47.919947 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:47.919953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:47.919972 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:52:47.920146 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:47.920151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:47.920216 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:47.920331 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:47.921729 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.922440 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.923738 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:47.924139 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:47.924283 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:47.929022 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:47.929039 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:47.929057 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:47.929103 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:47.931964 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:47.932702 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:47.932709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:47.932712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:47.932873 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:47.933318 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:47.933322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:47.933324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:47.933686 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:47.933938 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:47.934236 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 16:52:47.934433 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:52:47.935415 24967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:48.039849 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 24737
I20260812 16:52:48.067958 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 24983
Aug 12 16:52:48 dist-test-slave-z8x9 krb5kdc[24272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.195.131@KRBTEST.COM
I20260812 16:52:48.083995 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:48.068978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.8.195.131:38515)
0812 16:52:48.069466 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:48.069472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:48.069491 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:52:48.069552 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:48.069555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:48.069624 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:48.069724 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:48.071140 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:48.071871 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:48.073093 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:48.073771 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:48.073872 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:48.078384 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:48.078400 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:48.078415 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:48.078455 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:48.081365 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:48.082012 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:48.082018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:48.082021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:48.082168 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:48.082527 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:48.082530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:48.082532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:48.082897 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:48.083121 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:48.083432 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 16:52:48.083664 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 16:52:48.084690 25246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:48.238446 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 24983
I20260812 16:52:48.270383 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 24300
I20260812 16:52:48.271726 24430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:48.391800 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 24300
2026-08-12T16:52:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7757 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:48 dist-test-slave-z8x9 krb5kdc[25518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:48 dist-test-slave-z8x9 krb5kdc[25518](info): set up 2 sockets
Aug 12 16:52:48 dist-test-slave-z8x9 krb5kdc[25518](info): commencing operation
krb5kdc: starting...
W20260812 16:52:49.807695 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:50 dist-test-slave-z8x9 krb5kdc[25518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:50.119591 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41755
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:41755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:50.470726 25551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:50.471132 25551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:50.471318 25551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:50.479132 25551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:50.479306 25551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:50.479399 25551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:50.479501 25551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:50.479578 25551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:50.489880 25551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:41755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:50.491204 25551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:50.492503 25551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:50.504989 25558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:50.505016 25560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:50.505132 25563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:50.505864 25551 server_base.cc:1034] running on GCE node
I20260812 16:52:50.506681 25551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:50.508116 25551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:50.509342 25551 hybrid_clock.cc:648] HybridClock initialized: now 1786553570509281 us; error 49 us; skew 500 ppm
I20260812 16:52:50.509707 25551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T16:52:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2186 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:52:50 dist-test-slave-z8x9 krb5kdc[25574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:52:50 dist-test-slave-z8x9 krb5kdc[25574](info): set up 2 sockets
Aug 12 16:52:50 dist-test-slave-z8x9 krb5kdc[25574](info): commencing operation
krb5kdc: starting...
W20260812 16:52:52.005944 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:52:52 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:52:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:52:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:52:52.307271 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40249
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:52.653807 25600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:52.654201 25600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:52.654387 25600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:52.664013 25600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:52.664178 25600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:52.664264 25600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:52.664368 25600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:52.664440 25600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:52.676288 25600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:52.677743 25600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:52.679031 25600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:52.691283 25610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:52.691362 25607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:52.691321 25606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:52.692255 25600 server_base.cc:1034] running on GCE node
I20260812 16:52:52.693261 25600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:52.694800 25600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:52.696018 25600 hybrid_clock.cc:648] HybridClock initialized: now 1786553572695975 us; error 37 us; skew 500 ppm
I20260812 16:52:52.696385 25600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:52 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:52.699684 25600 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:52.701428 25600 webserver.cc:460] Webserver started at http://127.8.195.190:37345/ using document root <none> and password file <none>
I20260812 16:52:52.702140 25600 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:52.702284 25600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:52.702641 25600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:52.705932 25600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "e2c9bb94263c4f4f95026131a3eb0db7"
format_stamp: "Formatted at 2026-08-12 16:52:52 on dist-test-slave-z8x9"
server_key: "1ea5df6bc3c8f794cb89fd952ffcda0e"
server_key_iv: "172949f8850fc07c5faae71772eb82b7"
server_key_version: "encryptionkey@0"
I20260812 16:52:52.706799 25600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "e2c9bb94263c4f4f95026131a3eb0db7"
format_stamp: "Formatted at 2026-08-12 16:52:52 on dist-test-slave-z8x9"
server_key: "1ea5df6bc3c8f794cb89fd952ffcda0e"
server_key_iv: "172949f8850fc07c5faae71772eb82b7"
server_key_version: "encryptionkey@0"
I20260812 16:52:52.712174 25600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 16:52:52.716009 25622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:52.717458 25600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:52:52.717661 25600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "e2c9bb94263c4f4f95026131a3eb0db7"
format_stamp: "Formatted at 2026-08-12 16:52:52 on dist-test-slave-z8x9"
server_key: "1ea5df6bc3c8f794cb89fd952ffcda0e"
server_key_iv: "172949f8850fc07c5faae71772eb82b7"
server_key_version: "encryptionkey@0"
I20260812 16:52:52.717872 25600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:52.739857 25600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:52.744256 25600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:52.744589 25600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:52.764259 25600 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:40249
I20260812 16:52:52.764295 25713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:40249 every 8 connection(s)
I20260812 16:52:52.765755 25600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:52.766238 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 25600
I20260812 16:52:52.766467 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:52:52.767364 8974 external_mini_cluster.cc:1428] Setting key 348ff541e9e2ddbee1a3d7bf05d6f024
I20260812 16:52:52.770571 25714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:52:52.783595 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Bootstrap starting.
I20260812 16:52:52.786914 25714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Neither blocks nor log segments found. Creating new log.
Aug 12 16:52:52 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:52.788029 25714 log.cc:824] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:52.791363 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: No bootstrap required, opened a new log
I20260812 16:52:52.796558 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:52.769679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.8.195.190:40249)
0812 16:52:52.770567 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:52.770583 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:52.770638 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:52:52.773035 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:52.773063 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:52.773595 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:52.774087 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:52.775489 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:52.777619 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:52.778870 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:52.779910 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:52.780110 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:52.788435 (+ 8325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:52.788444 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:52.788459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:52.788530 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:52.792004 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:52.792737 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:52.792745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:52.792750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:52.792939 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:52.793409 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:52.793416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:52.793424 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:52.793999 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:52.794324 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:52.794808 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 16:52:52.795223 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":365,"threads_started":1}
I20260812 16:52:52.800707 25714 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:52.801093 25714 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:52:52.801223 25714 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c9bb94263c4f4f95026131a3eb0db7, State: Initialized, Role: FOLLOWER
I20260812 16:52:52.801803 25714 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:52.802047 25714 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:52.802201 25714 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:52.802403 25714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:52:52.804982 25714 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:52.805581 25714 leader_election.cc:302] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c9bb94263c4f4f95026131a3eb0db7; no voters:
I20260812 16:52:52.806169 25714 leader_election.cc:288] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:52:52.806331 25723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:52:52.807039 25723 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 LEADER]: Becoming Leader. State: Replica: e2c9bb94263c4f4f95026131a3eb0db7, State: Running, Role: LEADER
I20260812 16:52:52.807574 25723 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:52.808287 25714 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:52.810598 25724 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } } }
I20260812 16:52:52.810588 25726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c9bb94263c4f4f95026131a3eb0db7. Latest consensus state: current_term: 1 leader_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } } }
I20260812 16:52:52.811057 25724 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:52.811057 25726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:52.817544 25734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:52.824342 25734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:52.835397 25734 catalog_manager.cc:1347] Generated new cluster ID: 360ed95146184f17a221b062ff118bfa
I20260812 16:52:52.835525 25734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:52.870884 25734 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:52:52.872838 25734 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:52.880285 25734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Generated new TSK 0
I20260812 16:52:52.881138 25734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:52:52.959297 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:53.322842 25761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:53.323158 25761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:53.323364 25761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:53.330899 25761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:53.331051 25761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:53.331247 25761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:53.342569 25761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:53.344317 25761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:53.345674 25761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:53.359037 25774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:53.359068 25778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:53.359105 25773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:53.359897 25761 server_base.cc:1034] running on GCE node
I20260812 16:52:53.360590 25761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:53.361678 25761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:53.362902 25761 hybrid_clock.cc:648] HybridClock initialized: now 1786553573362853 us; error 34 us; skew 500 ppm
I20260812 16:52:53.363276 25761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:53 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:53.366838 25761 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:52:53.368484 25761 webserver.cc:460] Webserver started at http://127.8.195.129:33351/ using document root <none> and password file <none>
I20260812 16:52:53.369280 25761 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:53.369418 25761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:53.369762 25761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:53.372937 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "4f8110aa355e4219838fc5c2f81a07dd"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "9cfe843f4b0ea1d07c2f350d1e412325"
server_key_iv: "618c61b1b95f47dd782329213e9ef267"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.373819 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "4f8110aa355e4219838fc5c2f81a07dd"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "9cfe843f4b0ea1d07c2f350d1e412325"
server_key_iv: "618c61b1b95f47dd782329213e9ef267"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.379065 25761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:52:53.382980 25789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:53.384519 25761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 16:52:53.384738 25761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "4f8110aa355e4219838fc5c2f81a07dd"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "9cfe843f4b0ea1d07c2f350d1e412325"
server_key_iv: "618c61b1b95f47dd782329213e9ef267"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.384982 25761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:53.406181 25761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:53.411337 25761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:53.411669 25761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:53.412767 25761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:53.414213 25761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:53.414327 25761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:53.414484 25761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:53.414561 25761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:53.450716 25761 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:35273
I20260812 16:52:53.450814 25982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:35273 every 8 connection(s)
I20260812 16:52:53.452098 25761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:53.461148 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 25761
I20260812 16:52:53.461424 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:52:53.462266 8974 external_mini_cluster.cc:1428] Setting key b6d4ae1561248bfa56051f27346b090f
Aug 12 16:52:53 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:53.474344 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:53.455079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:47837 (local address 127.8.195.190:40249)
0812 16:52:53.455547 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:53.455552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:53.456508 (+ 956us) server_negotiation.cc:407] Connection header received
0812 16:52:53.458480 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:53.458485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:53.458559 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:53.458662 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:53.460227 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:53.461038 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:53.463384 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:53.463837 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:53.463986 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:53.468127 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:53.468134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:53.468138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:53.468175 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:53.470968 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:53.471777 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:53.471790 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:53.471793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:53.471920 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:53.472405 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:53.472409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:53.472411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:53.472707 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:53.472872 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:53.473723 (+ 851us) server_negotiation.cc:299] Negotiation successful
0812 16:52:53.473968 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":234,"threads_started":1}
I20260812 16:52:53.475509 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:53.455622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:47837)
0812 16:52:53.456327 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:53.456373 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:53.458180 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:53.458896 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:53.458905 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:53.459431 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:53.460012 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:53.460029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:53.461328 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:53.461335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:53.463234 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:53.463248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:53.464086 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:53.464094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:53.464493 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:53.465321 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:53.465351 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:53.467851 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:53.471129 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:53.471139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:53.471166 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:53.471629 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:53.472047 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:53.472052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:53.472056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:53.472268 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:53.472842 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:53.472851 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:53.473298 (+ 447us) client_negotiation.cc:769] Sending connection context
0812 16:52:53.473588 (+ 290us) client_negotiation.cc:240] Negotiation successful
0812 16:52:53.474172 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":266,"threads_started":1}
I20260812 16:52:53.477238 25983 heartbeater.cc:347] Connected to a master server at 127.8.195.190:40249
I20260812 16:52:53.477567 25983 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:53.478348 25983 heartbeater.cc:511] Master 127.8.195.190:40249 requested a full tablet report, sending...
I20260812 16:52:53.480605 25656 ts_manager.cc:194] Registered new tserver with Master: 4f8110aa355e4219838fc5c2f81a07dd (127.8.195.129:35273)
I20260812 16:52:53.482192 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:47837
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:52:53.530210 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:53.879210 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:53.879547 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:53.879724 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:53.887303 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:53.887502 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:53.887722 25996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:53.898006 25996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:53.899289 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:53.900661 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:53.914115 26010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:53.914144 26005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:53.914129 26007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:53.915218 25996 server_base.cc:1034] running on GCE node
I20260812 16:52:53.915807 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:53.916724 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:53.917984 25996 hybrid_clock.cc:648] HybridClock initialized: now 1786553573917914 us; error 52 us; skew 500 ppm
I20260812 16:52:53.918398 25996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:53 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:53.929121 25996 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:52:53.930927 25996 webserver.cc:460] Webserver started at http://127.8.195.130:34661/ using document root <none> and password file <none>
I20260812 16:52:53.931701 25996 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:53.931862 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:53.932209 25996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:53.935322 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "a53aa95337b8449fb53e7fdb7dd24006"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "edc087ffad3f861a8bc77f7caf79ecf9"
server_key_iv: "2b7822a707e978795ffd67963c7062fb"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.936141 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "a53aa95337b8449fb53e7fdb7dd24006"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "edc087ffad3f861a8bc77f7caf79ecf9"
server_key_iv: "2b7822a707e978795ffd67963c7062fb"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.941751 25996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:52:53.945465 26024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:53.946879 25996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:52:53.947081 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "a53aa95337b8449fb53e7fdb7dd24006"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "edc087ffad3f861a8bc77f7caf79ecf9"
server_key_iv: "2b7822a707e978795ffd67963c7062fb"
server_key_version: "encryptionkey@0"
I20260812 16:52:53.947319 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:54.004189 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:54.009265 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:54.009598 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:54.010703 25996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:54.012346 25996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:54.012455 25996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:54.012624 25996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:54.012701 25996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:54.048727 25996 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:35139
I20260812 16:52:54.048774 26241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:35139 every 8 connection(s)
I20260812 16:52:54.050153 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:54.052551 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 25996
I20260812 16:52:54.052716 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:52:54.053328 8974 external_mini_cluster.cc:1428] Setting key c7eaadd58715ac30a1ed55568553c6d3
Aug 12 16:52:54 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:54.072826 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.053370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43677 (local address 127.8.195.190:40249)
0812 16:52:54.053895 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:54.053903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:54.056961 (+ 3058us) server_negotiation.cc:407] Connection header received
0812 16:52:54.057018 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:54.057021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:54.057084 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:54.057241 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:54.058811 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.059555 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.061162 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.061655 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.061804 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.065899 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:54.065904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:54.065907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:54.065943 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:54.068753 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.069811 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.069816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.069818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.069964 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.070560 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.070568 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.070573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.071013 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:54.071202 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:54.072341 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0812 16:52:54.072530 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":257,"threads_started":1}
I20260812 16:52:54.074559 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.054007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:43677)
0812 16:52:54.054892 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:54.054939 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:54.056663 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:54.057505 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:54.057514 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:54.058039 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:54.058633 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:54.058650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.059712 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.059717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:54.060999 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:54.061012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.061922 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.061930 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:54.062184 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.062962 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:54.062986 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:54.065621 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:54.068917 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:54.068930 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:54.068951 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:54.069686 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:54.070091 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:54.070099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:54.070105 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:54.070394 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:54.071176 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:54.071219 (+ 43us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:54.071833 (+ 614us) client_negotiation.cc:769] Sending connection context
0812 16:52:54.072231 (+ 398us) client_negotiation.cc:240] Negotiation successful
0812 16:52:54.072871 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":371,"threads_started":1}
I20260812 16:52:54.076283 26242 heartbeater.cc:347] Connected to a master server at 127.8.195.190:40249
I20260812 16:52:54.076651 26242 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:54.077514 26242 heartbeater.cc:511] Master 127.8.195.190:40249 requested a full tablet report, sending...
I20260812 16:52:54.079142 25656 ts_manager.cc:194] Registered new tserver with Master: a53aa95337b8449fb53e7fdb7dd24006 (127.8.195.130:35139)
I20260812 16:52:54.080019 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:43677
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:52:54.121383 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:54.484748 26252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:54.485101 26252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:54.485322 26252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 16:52:54.485509 25983 heartbeater.cc:503] Master 127.8.195.190:40249 was elected leader, sending a full tablet report...
W20260812 16:52:54.493079 26252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:54.493266 26252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:54.493451 26252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:54.503294 26252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:54.504683 26252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:54.506081 26252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:54.519641 26260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:54.519706 26264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:54.519706 26261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:54.520437 26252 server_base.cc:1034] running on GCE node
I20260812 16:52:54.521139 26252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:54.522214 26252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:54.523451 26252 hybrid_clock.cc:648] HybridClock initialized: now 1786553574523395 us; error 46 us; skew 500 ppm
I20260812 16:52:54.523787 26252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:54 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553574, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:54.527141 26252 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:52:54.528698 26252 webserver.cc:460] Webserver started at http://127.8.195.131:38273/ using document root <none> and password file <none>
I20260812 16:52:54.529500 26252 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:54.529645 26252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:54.529968 26252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:52:54.532924 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "97f0ad8efc5f47a3814e2fd59195a58a"
format_stamp: "Formatted at 2026-08-12 16:52:54 on dist-test-slave-z8x9"
server_key: "7813bc7b907d765f097449cfd9d6f3c2"
server_key_iv: "377b55e0a24836ff09570cf7d76d7c2d"
server_key_version: "encryptionkey@0"
I20260812 16:52:54.533784 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "97f0ad8efc5f47a3814e2fd59195a58a"
format_stamp: "Formatted at 2026-08-12 16:52:54 on dist-test-slave-z8x9"
server_key: "7813bc7b907d765f097449cfd9d6f3c2"
server_key_iv: "377b55e0a24836ff09570cf7d76d7c2d"
server_key_version: "encryptionkey@0"
I20260812 16:52:54.539229 26252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:54.542861 26275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:54.544292 26252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:52:54.544495 26252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "97f0ad8efc5f47a3814e2fd59195a58a"
format_stamp: "Formatted at 2026-08-12 16:52:54 on dist-test-slave-z8x9"
server_key: "7813bc7b907d765f097449cfd9d6f3c2"
server_key_iv: "377b55e0a24836ff09570cf7d76d7c2d"
server_key_version: "encryptionkey@0"
I20260812 16:52:54.544721 26252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:54.569854 26252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:54.574216 26252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:54.574517 26252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:54.575487 26252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:54.576841 26252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:54.576944 26252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:54.577090 26252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:54.577215 26252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 16:52:54.612042 26252 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:43595
I20260812 16:52:54.612138 26496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:43595 every 8 connection(s)
I20260812 16:52:54.613492 26252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:54.618016 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 26252
I20260812 16:52:54.618201 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:52:54.618718 8974 external_mini_cluster.cc:1428] Setting key 52399651ba575c75235e63e5f3fcd9e8
Aug 12 16:52:54 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553574, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:52:54.634068 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.616159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:54005 (local address 127.8.195.190:40249)
0812 16:52:54.616604 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:54.616609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:54.617583 (+ 974us) server_negotiation.cc:407] Connection header received
0812 16:52:54.619893 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:54.619899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:54.620028 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:54.620149 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:54.621739 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.622595 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.624188 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.624605 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.624747 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.628301 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:54.628307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:54.628310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:54.628341 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:54.630924 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.631667 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.631671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.631674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.631795 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.632247 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.632251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.632254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.632511 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:54.632681 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:54.633405 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 16:52:54.633629 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260812 16:52:54.635116 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.616657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:54005)
0812 16:52:54.617422 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:54.617460 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:54.619615 (+ 2155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:54.620394 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:54.620403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:54.620906 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:54.621565 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:54.621597 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.622786 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.622791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:54.624024 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:54.624041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.624845 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.624854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:54.625114 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.625859 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:54.625881 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:54.628073 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:52:54.631072 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:54.631081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:54.631098 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:54.631526 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:54.631907 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:52:54.631912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:52:54.631916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:52:54.632106 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:52:54.632642 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:52:54.632651 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:52:54.632993 (+ 342us) client_negotiation.cc:769] Sending connection context
0812 16:52:54.633311 (+ 318us) client_negotiation.cc:240] Negotiation successful
0812 16:52:54.633864 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":273,"threads_started":1}
I20260812 16:52:54.636665 26498 heartbeater.cc:347] Connected to a master server at 127.8.195.190:40249
I20260812 16:52:54.637003 26498 heartbeater.cc:464] Registering TS with master...
I20260812 16:52:54.637799 26498 heartbeater.cc:511] Master 127.8.195.190:40249 requested a full tablet report, sending...
I20260812 16:52:54.639309 25656 ts_manager.cc:194] Registered new tserver with Master: 97f0ad8efc5f47a3814e2fd59195a58a (127.8.195.131:43595)
I20260812 16:52:54.640113 25656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:54005
I20260812 16:52:54.646903 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
Entry for principal oryx/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab.
WARNING: no policy specified for oryx/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
Entry for principal oryx/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab.
WARNING: no policy specified for oryx/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
Entry for principal oryx/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab.
I20260812 16:52:54.836642 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 25761
Aug 12 16:52:54 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:52:54.854195 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.838107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.8.195.129:35273)
0812 16:52:54.838747 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:54.838755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:54.838788 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:52:54.838859 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:54.838866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:54.839089 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:54.839239 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:54.840685 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.841838 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.843115 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.843672 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.843772 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.848122 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:54.848137 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:54.848152 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:54.848189 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:54.851144 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.851944 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.851951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.851954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.852103 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.852559 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.852562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.852564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.852937 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:54.853230 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:54.853562 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 16:52:54.853771 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":318,"threads_started":1}
I20260812 16:52:54.855525 25974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:54.940373 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 25761
I20260812 16:52:54.969024 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 25996
Aug 12 16:52:54 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:52:54.986187 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:54.970223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.8.195.130:35139)
0812 16:52:54.970737 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:54.970744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:54.970770 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:52:54.970829 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:54.970836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:54.971039 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:54.971217 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:54.972544 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.973654 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.974969 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:54.975550 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:54.975696 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:54.980230 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:54.980243 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:54.980258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:54.980315 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:54.983243 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.983968 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.983977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.983979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.984110 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:54.984489 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:54.984495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:54.984498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:54.984858 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:54.985216 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:54.985539 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 16:52:54.985821 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":273,"threads_started":1}
I20260812 16:52:54.987594 26233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:55.071594 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 25996
I20260812 16:52:55.097568 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26252
Aug 12 16:52:55 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:52:55.114954 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:55.098682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.8.195.131:43595)
0812 16:52:55.099235 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:55.099242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:55.099272 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:52:55.099333 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:55.099349 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:55.099565 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:55.099742 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:55.101149 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:55.102228 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:55.103515 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:55.104088 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:55.104253 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:55.108941 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:55.108954 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:55.108968 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:55.109004 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:55.112035 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:55.112712 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:55.112719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:55.112722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:55.112849 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:55.113266 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:55.113274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:55.113277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:55.113646 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:55.113978 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:55.114277 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 16:52:55.114563 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":266,"threads_started":1}
I20260812 16:52:55.116307 26489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:55.200229 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 26252
I20260812 16:52:55.232769 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 25600
I20260812 16:52:55.234102 25708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:52:55.339970 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 25600
I20260812 16:52:55.367859 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40249
--webserver_interface=127.8.195.190
--webserver_port=37345
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:52:55.720419 26525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:55.720744 26525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:55.720839 26525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:55.728175 26525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:52:55.728307 26525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:55.728356 26525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:55.728410 26525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:52:55.728456 26525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:52:55.738685 26525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=37345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:55.740085 26525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:55.741477 26525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:55.754339 26533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:55.754416 26535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:55.754423 26538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:55.755609 26525 server_base.cc:1034] running on GCE node
I20260812 16:52:55.756613 26525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:55.758394 26525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:55.759686 26525 hybrid_clock.cc:648] HybridClock initialized: now 1786553575759626 us; error 72 us; skew 500 ppm
I20260812 16:52:55.760116 26525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:55 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553575, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:55.763518 26525 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:52:55.765141 26525 webserver.cc:460] Webserver started at http://127.8.195.190:37345/ using document root <none> and password file <none>
I20260812 16:52:55.765960 26525 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:55.766139 26525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:55.772094 26525 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 16:52:55.775163 26552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:55.776530 26525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:52:55.776743 26525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "e2c9bb94263c4f4f95026131a3eb0db7"
format_stamp: "Formatted at 2026-08-12 16:52:52 on dist-test-slave-z8x9"
server_key: "1ea5df6bc3c8f794cb89fd952ffcda0e"
server_key_iv: "172949f8850fc07c5faae71772eb82b7"
server_key_version: "encryptionkey@0"
I20260812 16:52:55.777840 26525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:55.796510 26525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:55.800909 26525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:55.801283 26525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:55.820801 26525 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:40249
I20260812 16:52:55.820840 26655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:40249 every 8 connection(s)
I20260812 16:52:55.822413 26525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:52:55.827808 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 26525
I20260812 16:52:55.828660 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:35273
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:40249
--webserver_port=33351
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:52:55.829761 26656 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:52:55.833827 26656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Bootstrap starting.
I20260812 16:52:55.855605 26656 log.cc:824] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Log is configured to *not* fsync() on all Append() calls
I20260812 16:52:55.865530 26656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:52:55.866153 26656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Bootstrap complete.
I20260812 16:52:55.876451 26656 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:55.877553 26656 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c9bb94263c4f4f95026131a3eb0db7, State: Initialized, Role: FOLLOWER
I20260812 16:52:55.878252 26656 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:55.878513 26656 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:52:55.878657 26656 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:52:55.878881 26656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:52:55.883412 26656 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:55.883997 26656 leader_election.cc:302] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2c9bb94263c4f4f95026131a3eb0db7; no voters:
I20260812 16:52:55.884727 26656 leader_election.cc:288] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:52:55.884958 26664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:52:55.886372 26664 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [term 2 LEADER]: Becoming Leader. State: Replica: e2c9bb94263c4f4f95026131a3eb0db7, State: Running, Role: LEADER
I20260812 16:52:55.887235 26656 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:52:55.887161 26664 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } }
I20260812 16:52:55.896109 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c9bb94263c4f4f95026131a3eb0db7. Latest consensus state: current_term: 2 leader_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } } }
I20260812 16:52:55.896554 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:55.897332 26665 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2c9bb94263c4f4f95026131a3eb0db7" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40249 } } }
I20260812 16:52:55.897748 26665 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7 [sys.catalog]: This master's current role is: LEADER
I20260812 16:52:55.911530 26686 catalog_manager.cc:1259] Loaded cluster ID: 360ed95146184f17a221b062ff118bfa
I20260812 16:52:55.911655 26686 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: loading cluster ID for follower catalog manager: success
I20260812 16:52:55.915732 26686 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: acquiring CA information for follower catalog manager: success
I20260812 16:52:55.918294 26686 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:52:55.918543 26693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:52:55.920269 26693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:52:55.921252 26693 catalog_manager.cc:1259] Loaded cluster ID: 360ed95146184f17a221b062ff118bfa
I20260812 16:52:55.921368 26693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:52:55.922596 26693 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:52:55.923514 26693 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2c9bb94263c4f4f95026131a3eb0db7: Loaded TSK: 0
I20260812 16:52:55.924292 26693 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:52:56.213622 26659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:56.214040 26659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:56.214217 26659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:56.222223 26659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:56.222427 26659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:56.222615 26659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:52:56.233281 26659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.129.keytab
--principal=oryx/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:35273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=33351
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:56.234822 26659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:56.236256 26659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:56.252882 26708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:56.253053 26706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:56.253729 26704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:56.254065 26659 server_base.cc:1034] running on GCE node
I20260812 16:52:56.254849 26659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:56.257154 26659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:56.258548 26659 hybrid_clock.cc:648] HybridClock initialized: now 1786553576258461 us; error 64 us; skew 500 ppm
I20260812 16:52:56.258977 26659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553576, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:56.263305 26659 init.cc:377] Logged in from keytab as oryx/127.8.195.129@KRBTEST.COM (short username oryx)
I20260812 16:52:56.265257 26659 webserver.cc:460] Webserver started at http://127.8.195.129:33351/ using document root <none> and password file <none>
I20260812 16:52:56.266068 26659 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:56.266263 26659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:56.272667 26659 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 16:52:56.276096 26729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.277810 26659 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 16:52:56.278002 26659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "4f8110aa355e4219838fc5c2f81a07dd"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "9cfe843f4b0ea1d07c2f350d1e412325"
server_key_iv: "618c61b1b95f47dd782329213e9ef267"
server_key_version: "encryptionkey@0"
I20260812 16:52:56.279060 26659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:56.293435 26659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:56.298269 26659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:56.298592 26659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:56.299698 26659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:56.301229 26659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:56.301330 26659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.301470 26659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:56.301574 26659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.336633 26659 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:35273
I20260812 16:52:56.336684 26947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:35273 every 8 connection(s)
I20260812 16:52:56.338225 26659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:52:56.341087 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 26659
I20260812 16:52:56.341809 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:35139
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:40249
--webserver_port=34661
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:56.705479 26952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:56.705781 26952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:56.705930 26952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:56.713318 26952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:56.713443 26952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:56.713639 26952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:52:56.723623 26952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.130.keytab
--principal=oryx/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:35139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=34661
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:56.725052 26952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:56.726416 26952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:56.750398 26962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:56.750826 26961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:56.751513 26969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:56.752167 26952 server_base.cc:1034] running on GCE node
I20260812 16:52:56.753437 26952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:56.755275 26952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:56.756603 26952 hybrid_clock.cc:648] HybridClock initialized: now 1786553576756488 us; error 106 us; skew 500 ppm
I20260812 16:52:56.757252 26952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553576, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:56.762857 26952 init.cc:377] Logged in from keytab as oryx/127.8.195.130@KRBTEST.COM (short username oryx)
I20260812 16:52:56.764868 26952 webserver.cc:460] Webserver started at http://127.8.195.130:34661/ using document root <none> and password file <none>
I20260812 16:52:56.765703 26952 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:56.765858 26952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:56.772593 26952 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:52:56.775959 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.777606 26952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:52:56.777805 26952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "a53aa95337b8449fb53e7fdb7dd24006"
format_stamp: "Formatted at 2026-08-12 16:52:53 on dist-test-slave-z8x9"
server_key: "edc087ffad3f861a8bc77f7caf79ecf9"
server_key_iv: "2b7822a707e978795ffd67963c7062fb"
server_key_version: "encryptionkey@0"
I20260812 16:52:56.778861 26952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:56.815304 26952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:56.820302 26952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:56.820627 26952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:56.821897 26952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:56.823484 26952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:56.823619 26952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.823768 26952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:56.823859 26952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:56.870040 26952 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:35139
I20260812 16:52:56.870083 27181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:35139 every 8 connection(s)
I20260812 16:52:56.871680 26952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:52:56.881673 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 26952
I20260812 16:52:56.882591 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:43595
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:40249
--webserver_port=38273
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:56 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:57.558003 26951 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.202s user 0.003s sys 0.011s
I20260812 16:52:57.560225 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:56.341428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:37803)
0812 16:52:56.343081 (+ 1653us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:56.343144 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:56.345906 (+ 2762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:56.347413 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:56.347426 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:56.348249 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:56.349075 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:56.349106 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.350771 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.350779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:56.352966 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:56.352989 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.354642 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.354647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:56.354849 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:56.355624 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:56.355648 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:57.558920 (+1203272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1309,"thread_start_us":1067,"threads_started":1}
W20260812 16:52:57.560225 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:56.341447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:37803 (local address 127.8.195.190:40249)
0812 16:52:56.342253 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:56.342271 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:56.343303 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 16:52:56.346513 (+ 3210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:56.346540 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:56.346893 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:56.347194 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:56.349338 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.350590 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.353215 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.354409 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.354635 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.559373 (+1204738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:37803: BlockingRecv error: recv got EOF from 127.8.195.129:37803 (error 108)
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":319,"threads_started":1}
W20260812 16:52:57.561694 26948 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:40249 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.195.190:40249: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:57.572150 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:57.562554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:36669)
0812 16:52:57.562847 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:57.562864 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:57.563013 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:57.563366 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:57.563371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:57.563794 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:57.563995 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.564005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.565066 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.565072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.566194 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.566208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.566973 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.566979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.567156 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.567854 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:57.567872 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:57.571790 (+ 3918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175}
W20260812 16:52:57.572289 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:57.562618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36669 (local address 127.8.195.190:40249)
0812 16:52:57.562795 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:57.562802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:57.562939 (+ 137us) server_negotiation.cc:407] Connection header received
0812 16:52:57.563151 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:57.563155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:57.563220 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:57.563332 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:57.564168 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.564907 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.566343 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.566775 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.566924 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.572033 (+ 5109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:36669: BlockingRecv error: recv got EOF from 127.8.195.129:36669 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:57.582427 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:57.573262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:52697)
0812 16:52:57.573592 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:57.573611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:57.573744 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:57.574119 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:57.574124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:57.574525 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:57.574691 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.574700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.575738 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.575743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.576716 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.576727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.577539 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.577545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.577763 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.578410 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:57.578427 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:57.582153 (+ 3726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227}
W20260812 16:52:57.582842 26948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:52:57.583518 26950 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:57.573423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:52697 (local address 127.8.195.190:40249)
0812 16:52:57.573580 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:57.573586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:57.573664 (+ 78us) server_negotiation.cc:407] Connection header received
0812 16:52:57.573878 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:57.573883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:57.573970 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:57.574080 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:57.574841 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.575577 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.576860 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.577379 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.577507 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.583324 (+ 5817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:52697: BlockingRecv error: recv got EOF from 127.8.195.129:52697 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 16:52:57.648761 27187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:52:57.649438 27187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:52:57.649693 27187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:52:57.669294 27187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:52:57.669559 27187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:52:57.669883 27187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:52:57.696091 27187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/krb5kdc/oryx_127.8.195.131.keytab
--principal=oryx/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:43595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=38273
--tserver_master_addrs=127.8.195.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:52:57.698745 27187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:52:57.701295 27187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:52:57.721946 27201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:57.722074 27203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:52:57.722785 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:52:57.723699 27187 server_base.cc:1034] running on GCE node
I20260812 16:52:57.724969 27187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:52:57.726897 27187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:52:57.728238 27187 hybrid_clock.cc:648] HybridClock initialized: now 1786553577728138 us; error 65 us; skew 500 ppm
I20260812 16:52:57.728847 27187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553577, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:52:57.734489 27187 init.cc:377] Logged in from keytab as oryx/127.8.195.131@KRBTEST.COM (short username oryx)
I20260812 16:52:57.736279 27187 webserver.cc:460] Webserver started at http://127.8.195.131:38273/ using document root <none> and password file <none>
I20260812 16:52:57.737053 27187 fs_manager.cc:359] Metadata directory not provided
I20260812 16:52:57.737253 27187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:52:57.743711 27187 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 16:52:57.747334 27227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:57.749310 27187 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 16:52:57.749549 27187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "97f0ad8efc5f47a3814e2fd59195a58a"
format_stamp: "Formatted at 2026-08-12 16:52:54 on dist-test-slave-z8x9"
server_key: "7813bc7b907d765f097449cfd9d6f3c2"
server_key_iv: "377b55e0a24836ff09570cf7d76d7c2d"
server_key_version: "encryptionkey@0"
I20260812 16:52:57.750644 27187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:52:57.779042 27187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:52:57.784644 27187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:52:57.784969 27187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:52:57.786456 27187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:52:57.788375 27187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:52:57.788518 27187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:57.788659 27187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:52:57.788753 27187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:52:57.834617 27187 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:43595
I20260812 16:52:57.834682 27428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:43595 every 8 connection(s)
I20260812 16:52:57.836191 27187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:52:57.836659 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 27187
I20260812 16:52:57.873268 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:57.837959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.8.195.190:40249)
0812 16:52:57.838195 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:57.838204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:57.850965 (+ 12761us) server_negotiation.cc:407] Connection header received
0812 16:52:57.851238 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:57.851244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:57.851339 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:57.851460 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:57.853947 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.854801 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.856362 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.856829 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.856983 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.863970 (+ 6987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:52:57.863978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:52:57.863994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:52:57.864041 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:52:57.868341 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:57.869423 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:57.869432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:57.869441 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:57.869716 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:52:57.870215 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:52:57.870220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:52:57.870227 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:52:57.870819 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:52:57.871173 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 16:52:57.872377 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0812 16:52:57.872828 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:57 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:58.091722 27186 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.197s user 0.004s sys 0.011s
I20260812 16:52:58.093514 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:56.876195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:58479)
0812 16:52:56.877340 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:56.877429 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:56.881372 (+ 3943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:56.882803 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:56.882824 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:56.884192 (+ 1368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:56.885572 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:56.885617 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.887534 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.887544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:56.891088 (+ 3544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:56.891122 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.892601 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.892613 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:56.893154 (+ 541us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:56.894434 (+ 1280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:56.894498 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:58.092527 (+1198029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":369,"threads_started":1}
W20260812 16:52:58.093690 27185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:56.875567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:58479 (local address 127.8.195.190:40249)
0812 16:52:56.876278 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:56.876289 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:56.877634 (+ 1345us) server_negotiation.cc:407] Connection header received
0812 16:52:56.881990 (+ 4356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:56.881997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:56.882143 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:56.882280 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:56.885937 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.887248 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.891388 (+ 4140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:56.892216 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:56.892410 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.093069 (+1200659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:58479: BlockingRecv error: recv got EOF from 127.8.195.130:58479 (error 108)
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":308,"threads_started":1}
W20260812 16:52:58.094579 27182 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:40249 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.195.190:40249: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:58.107785 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:58.095288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:44691)
0812 16:52:58.095592 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:58.095623 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:58.095819 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:58.096613 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:58.096621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:58.097156 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:58.097391 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.097401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.099631 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.099636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.100656 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.100670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.102050 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.102056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.102180 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.103109 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:58.103128 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:58.107558 (+ 4430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 16:52:58.108397 27185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:58.095407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:44691 (local address 127.8.195.190:40249)
0812 16:52:58.096157 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:58.096167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:58.096194 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:52:58.096280 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:58.096288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:58.096401 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:58.096574 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:58.097614 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.099435 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.100825 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.101846 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.102037 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.108028 (+ 5991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:44691: BlockingRecv error: recv got EOF from 127.8.195.130:44691 (error 108)
Metrics: {"server-negotiator.queue_time_us":568}
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:58.121274 27185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:58.108915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:44427 (local address 127.8.195.190:40249)
0812 16:52:58.109251 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:58.109262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:58.109294 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:52:58.109461 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:58.109470 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:58.109588 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:58.109756 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:58.110859 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.112672 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.114288 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.115336 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.115519 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.121010 (+ 5491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:44427: BlockingRecv error: recv got EOF from 127.8.195.130:44427 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
I20260812 16:52:58.121313 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:58.108581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:44427)
0812 16:52:58.109094 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:58.109117 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:58.109263 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:58.109773 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:58.109778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:58.110466 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:58.110669 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.110679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.112968 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.112973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.114116 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.114136 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.115562 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.115567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.115726 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.116517 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:58.116535 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:58.120837 (+ 4302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387}
W20260812 16:52:58.121841 27182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:58 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 16:52:58.605558 27185 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:58.584276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:60327 (local address 127.8.195.190:40249)
0812 16:52:58.584555 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:58.584565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:58.587644 (+ 3079us) server_negotiation.cc:407] Connection header received
0812 16:52:58.588640 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:58.588649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:58.588785 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:58.588971 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:58.590481 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.592105 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.595045 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.595992 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.596175 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.605129 (+ 8954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:60327: BlockingRecv error: recv got EOF from 127.8.195.129:60327 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 16:52:58.606213 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:58.583732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:60327)
0812 16:52:58.587456 (+ 3724us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:58.587490 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:58.588430 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:58.589022 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:58.589030 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:58.589910 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:58.590196 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.590208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.592306 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.592315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.594782 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:58.594811 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:58.596287 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:58.596297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:58.596600 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:58.597850 (+ 1250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:58.597892 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:58.605354 (+ 7462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3460,"thread_start_us":3242,"threads_started":1}
W20260812 16:52:59.003938 27436 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.132s user 0.001s sys 0.015s
W20260812 16:52:59.005390 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:57.842178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:49675 (local address 127.8.195.190:40249)
0812 16:52:57.857668 (+ 15490us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:57.857675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:57.858500 (+ 825us) server_negotiation.cc:407] Connection header received
0812 16:52:57.860607 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:57.860612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:57.860699 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:57.860829 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:57.863275 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.865341 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.867946 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.868674 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.868802 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.004966 (+1136164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:49675: BlockingRecv error: recv got EOF from 127.8.195.131:49675 (error 108)
Metrics: {"server-negotiator.queue_time_us":15315,"thread_start_us":15131,"threads_started":1}
I20260812 16:52:59.005842 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:57.841289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:49675)
0812 16:52:57.858318 (+ 17029us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:57.858375 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:57.860304 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:57.861128 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:57.861147 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:57.862057 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:57.863013 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.863045 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.865561 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.865568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.867714 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:57.867732 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:57.869137 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:57.869158 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:57.869548 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:57.871370 (+ 1822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:57.871405 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:59.004756 (+1133351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16625,"thread_start_us":16332,"threads_started":1}
W20260812 16:52:59.006824 27429 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:40249 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.195.190:40249: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:59.017584 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:59.007551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:46193)
0812 16:52:59.007777 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:59.007794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:59.007911 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:59.008200 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:59.008204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:59.008592 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:59.008763 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.008772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.009937 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.009941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.010965 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.010977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.011834 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.011839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.011982 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.012742 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:59.012762 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:59.017225 (+ 4463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260812 16:52:59.017748 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:59.007660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:46193 (local address 127.8.195.190:40249)
0812 16:52:59.007918 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:59.007922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:59.007941 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:52:59.007999 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:59.008002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:59.008065 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:59.008151 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:59.008921 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.009787 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.011137 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.011640 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.011771 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.017499 (+ 5728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:46193: BlockingRecv error: recv got EOF from 127.8.195.131:46193 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:59.028191 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:59.018413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:50671)
0812 16:52:59.018609 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:59.018630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:59.018738 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:59.019081 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:59.019084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:59.019514 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:59.019704 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.019714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.020801 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.020805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.022002 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.022016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.022818 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.022824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.022978 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.023648 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:59.023669 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:59.027859 (+ 4190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 16:52:59.028380 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:59.018621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:50671 (local address 127.8.195.190:40249)
0812 16:52:59.018801 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:59.018805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:59.018820 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:52:59.018870 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:59.018874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:59.018943 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:59.019027 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:59.019875 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.020636 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.022170 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.022623 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.022747 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.028080 (+ 5333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:50671: BlockingRecv error: recv got EOF from 127.8.195.131:50671 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 16:52:59.028833 27429 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:59.132190 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:59.122649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:48901)
0812 16:52:59.123212 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:59.123230 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:59.123388 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:59.123781 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:59.123784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:59.124218 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:59.124395 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.124405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.125510 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.125514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.126506 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.126519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.127328 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.127332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.127464 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.128095 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:59.128113 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:59.131946 (+ 3833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":337,"threads_started":1}
W20260812 16:52:59.132411 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:59.122737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:48901 (local address 127.8.195.190:40249)
0812 16:52:59.122989 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:59.122998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:59.123316 (+ 318us) server_negotiation.cc:407] Connection header received
0812 16:52:59.123534 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:59.123538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:59.123625 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:59.123746 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:59.124553 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.125370 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.126670 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.127123 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.127280 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.132150 (+ 4870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:48901: BlockingRecv error: recv got EOF from 127.8.195.130:48901 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:52:59 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:52:59.618289 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:52:59.607583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:44931)
0812 16:52:59.608211 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 16:52:59.608228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:52:59.608410 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:52:59.608818 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:52:59.608823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:52:59.609435 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:52:59.609743 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.609757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.611155 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.611161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.612181 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:52:59.612203 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.612999 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.613005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:52:59.613240 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.613993 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:52:59.614013 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:52:59.617900 (+ 3887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":371,"threads_started":1}
W20260812 16:52:59.618449 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:52:59.607750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:44931 (local address 127.8.195.190:40249)
0812 16:52:59.608021 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 16:52:59.608030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:52:59.608332 (+ 302us) server_negotiation.cc:407] Connection header received
0812 16:52:59.608569 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:52:59.608573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:52:59.608639 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:52:59.608782 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:52:59.609903 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.610994 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.612348 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:52:59.612821 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:52:59.612973 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:52:59.618174 (+ 5201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:44931: BlockingRecv error: recv got EOF from 127.8.195.129:44931 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:00.039336 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:00.029541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:37051)
0812 16:53:00.030133 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:00.030155 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:00.030352 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:00.030729 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:00.030734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:00.031204 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:00.031369 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.031378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.032421 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.032431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.033468 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.033482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.034295 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.034302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.034479 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.035137 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:00.035155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:00.038934 (+ 3779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":352,"threads_started":1}
W20260812 16:53:00.039464 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:00.029645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:37051 (local address 127.8.195.190:40249)
0812 16:53:00.029927 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:00.029936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:00.030257 (+ 321us) server_negotiation.cc:407] Connection header received
0812 16:53:00.030489 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:00.030495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:00.030579 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:00.030689 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:00.031524 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.032265 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.033631 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.034090 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.034236 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.039203 (+ 4967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:37051: BlockingRecv error: recv got EOF from 127.8.195.131:37051 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:00.143545 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:00.133371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:43801)
0812 16:53:00.133876 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:00.133894 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:00.134018 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:00.134403 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:00.134407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:00.134839 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:00.135029 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.135038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.136195 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.136200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.137254 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.137268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.138050 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.138055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.138199 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.138895 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:00.138914 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:00.143180 (+ 4266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":289,"threads_started":1}
W20260812 16:53:00.143760 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:00.133413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43801 (local address 127.8.195.190:40249)
0812 16:53:00.133662 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:00.133677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:00.133984 (+ 307us) server_negotiation.cc:407] Connection header received
0812 16:53:00.134177 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:00.134182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:00.134254 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:00.134354 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:00.135191 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.136025 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.137397 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.137877 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.137986 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.143421 (+ 5435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:43801: BlockingRecv error: recv got EOF from 127.8.195.130:43801 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:00 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:00.629647 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:00.619432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:47645)
0812 16:53:00.620029 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:00.620050 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:00.620187 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:00.620488 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:00.620491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:00.620862 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:00.621031 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.621040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.622154 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.622185 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.623304 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:00.623319 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.624185 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.624189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:00.624334 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.625094 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:00.625120 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:00.629264 (+ 4144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":330,"threads_started":1}
W20260812 16:53:00.629712 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:00.619672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:47645 (local address 127.8.195.190:40249)
0812 16:53:00.619885 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:00.619889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:00.620181 (+ 292us) server_negotiation.cc:407] Connection header received
0812 16:53:00.620280 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:00.620284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:00.620352 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:00.620447 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:00.621231 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.621991 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.623474 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:00.624034 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:00.624177 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:00.629487 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:47645: BlockingRecv error: recv got EOF from 127.8.195.129:47645 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:01.051240 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:01.040644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:53075)
0812 16:53:01.041302 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:01.041328 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:01.041511 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:01.041909 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:01.041913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:01.042425 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:01.042645 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.042655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.043689 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.043693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.044746 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.044760 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.045768 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.045775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.045988 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.046771 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:01.046790 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:01.050882 (+ 4092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":326,"threads_started":1}
W20260812 16:53:01.051345 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:01.040729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:53075 (local address 127.8.195.190:40249)
0812 16:53:01.040994 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:01.041004 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:01.041489 (+ 485us) server_negotiation.cc:407] Connection header received
0812 16:53:01.041640 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:01.041644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:01.041754 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:01.041840 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:01.042817 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.043554 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.044893 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.045540 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.045707 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.051104 (+ 5397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:53075: BlockingRecv error: recv got EOF from 127.8.195.131:53075 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:01.154958 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:01.144960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:35645)
0812 16:53:01.145655 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:01.145673 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:01.145866 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:01.146318 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:01.146323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:01.146840 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:01.147102 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.147167 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.148278 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.148283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.149328 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.149341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.150035 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.150040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.150230 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.150901 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:01.150920 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:01.154615 (+ 3695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":356,"threads_started":1}
W20260812 16:53:01.155041 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:01.145148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:35645 (local address 127.8.195.190:40249)
0812 16:53:01.145406 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:01.145415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:01.145811 (+ 396us) server_negotiation.cc:407] Connection header received
0812 16:53:01.146014 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:01.146020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:01.146127 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:01.146239 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:01.147314 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.148127 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.149461 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.149897 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.149991 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.154864 (+ 4873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:35645: BlockingRecv error: recv got EOF from 127.8.195.130:35645 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:01 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:01.640417 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:01.630998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:54527)
0812 16:53:01.631617 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:01.631652 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:01.631822 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:01.632192 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:01.632197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:01.632766 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:01.632999 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.633016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.634068 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.634072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.635125 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:01.635143 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.635829 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.635833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:01.635965 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.636573 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:01.636592 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:01.640158 (+ 3566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":355,"threads_started":1}
W20260812 16:53:01.640625 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:01.631153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:54527 (local address 127.8.195.190:40249)
0812 16:53:01.631395 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:01.631402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:01.631769 (+ 367us) server_negotiation.cc:407] Connection header received
0812 16:53:01.631968 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:01.631971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:01.632033 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:01.632115 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:01.633133 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.633933 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.635266 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:01.635678 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:01.635776 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:01.640380 (+ 4604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:54527: BlockingRecv error: recv got EOF from 127.8.195.129:54527 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:02.062363 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:02.052605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:34597)
0812 16:53:02.053368 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:02.053398 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:02.053549 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:02.053934 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:02.053937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:02.054297 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:02.054493 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.054502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.055612 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.055617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.056660 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.056673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.057469 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.057473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.057634 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.058280 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:02.058298 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:02.062103 (+ 3805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":409,"threads_started":1}
W20260812 16:53:02.062606 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:02.052670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:34597 (local address 127.8.195.190:40249)
0812 16:53:02.052986 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:02.052997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:02.053489 (+ 492us) server_negotiation.cc:407] Connection header received
0812 16:53:02.053710 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:02.053714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:02.053797 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:02.053886 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:02.054628 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.055430 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.056796 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.057316 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.057413 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.062346 (+ 4933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:34597: BlockingRecv error: recv got EOF from 127.8.195.131:34597 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:02.166254 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:02.156224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:57841)
0812 16:53:02.156769 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:02.156785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:02.156950 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:02.157386 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:02.157391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:02.157842 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:02.158015 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.158024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.159140 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.159144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.160179 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.160191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.160959 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.160965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.161162 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.161882 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:02.161900 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:02.165883 (+ 3983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":308,"threads_started":1}
W20260812 16:53:02.166416 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:02.156328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:57841 (local address 127.8.195.190:40249)
0812 16:53:02.156568 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:02.156577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:02.156878 (+ 301us) server_negotiation.cc:407] Connection header received
0812 16:53:02.157080 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:02.157086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:02.157210 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:02.157353 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:02.158167 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.158950 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.160337 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.160803 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.160938 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.166127 (+ 5189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:57841: BlockingRecv error: recv got EOF from 127.8.195.130:57841 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:02 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:02.650993 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:02.641606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:44839)
0812 16:53:02.642197 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:02.642215 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:02.642344 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:02.642688 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:02.642692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:02.643139 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:02.643308 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.643318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.644350 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.644355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.645373 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:02.645387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.646185 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.646190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:02.646362 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.647076 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:02.647095 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:02.650763 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":328,"threads_started":1}
W20260812 16:53:02.651254 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:02.641718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:44839 (local address 127.8.195.190:40249)
0812 16:53:02.641939 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:02.641944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:02.642330 (+ 386us) server_negotiation.cc:407] Connection header received
0812 16:53:02.642439 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:02.642443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:02.642513 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:02.642612 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:02.643449 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.644203 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.645507 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:02.646018 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:02.646123 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:02.650982 (+ 4859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:44839: BlockingRecv error: recv got EOF from 127.8.195.129:44839 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:03.087702 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:03.063638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:36723)
0812 16:53:03.069694 (+ 6056us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:03.069718 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:03.069871 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:03.070335 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:03.070339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:03.070886 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:03.071125 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.071146 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.073480 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.073488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.075127 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.075148 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.076589 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.076596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.076769 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.083342 (+ 6573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:03.083377 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:03.087434 (+ 4057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5937,"thread_start_us":5809,"threads_started":1}
W20260812 16:53:03.087926 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:03.064013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:36723 (local address 127.8.195.190:40249)
0812 16:53:03.064252 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:03.064259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:03.069850 (+ 5591us) server_negotiation.cc:407] Connection header received
0812 16:53:03.069991 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:03.069999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:03.070119 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:03.070273 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:03.071352 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.073301 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.075334 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.076384 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.076585 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.087666 (+ 11081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:36723: BlockingRecv error: recv got EOF from 127.8.195.131:36723 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:03.176973 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:03.167309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:43193)
0812 16:53:03.167960 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:03.167986 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:03.168156 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:03.168510 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:03.168517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:03.168996 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:03.169156 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.169218 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.170324 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.170330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.171281 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.171294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.172016 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.172023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.172216 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.172896 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:03.172915 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:03.176724 (+ 3809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":368,"threads_started":1}
W20260812 16:53:03.177335 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:03.167679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43193 (local address 127.8.195.190:40249)
0812 16:53:03.167919 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:03.167926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:03.168080 (+ 154us) server_negotiation.cc:407] Connection header received
0812 16:53:03.168275 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:03.168279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:03.168343 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:03.168466 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:03.169363 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.170164 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.171444 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.171861 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.172000 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.176987 (+ 4987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:43193: BlockingRecv error: recv got EOF from 127.8.195.130:43193 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:03 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:03.662351 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:03.652010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:45461)
0812 16:53:03.652591 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:03.652607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:03.652787 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:03.653132 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:03.653137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:03.653621 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:03.653816 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.653826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.655188 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.655196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.656283 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:03.656297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.657109 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.657115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:03.657358 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.658079 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:03.658098 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:03.662000 (+ 3902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":320,"threads_started":1}
W20260812 16:53:03.662600 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:03.652186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:45461 (local address 127.8.195.190:40249)
0812 16:53:03.652436 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:03.652443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:03.652696 (+ 253us) server_negotiation.cc:407] Connection header received
0812 16:53:03.652918 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:03.652928 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:03.652991 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:03.653096 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:03.653962 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.655038 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.656432 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:03.656941 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:03.657083 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:03.662290 (+ 5207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:45461: BlockingRecv error: recv got EOF from 127.8.195.129:45461 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:04.099279 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:04.088799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:38851)
0812 16:53:04.089421 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:04.089444 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:04.089567 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:04.089918 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:04.089921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:04.090469 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:04.090624 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.090633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.091898 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.091904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.093216 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.093232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.094348 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.094354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.094505 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.095121 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:04.095139 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:04.099009 (+ 3870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":348,"threads_started":1}
W20260812 16:53:04.099607 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:04.088881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:38851 (local address 127.8.195.190:40249)
0812 16:53:04.089059 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:04.089064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:04.089564 (+ 500us) server_negotiation.cc:407] Connection header received
0812 16:53:04.089676 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:04.089681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:04.089772 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:04.089874 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:04.090795 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.091747 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.093364 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.094071 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.094228 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.099255 (+ 5027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:38851: BlockingRecv error: recv got EOF from 127.8.195.131:38851 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:04.187938 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:04.178131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:58353)
0812 16:53:04.178657 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:04.178673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:04.178845 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:04.179184 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:04.179189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:04.179686 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:04.179864 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.179874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.180962 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.180968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.181985 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.181998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.182817 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.182823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.183018 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.183894 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:04.183914 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:04.187592 (+ 3678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
W20260812 16:53:04.188293 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:04.178224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:58353 (local address 127.8.195.190:40249)
0812 16:53:04.178472 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:04.178480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:04.178765 (+ 285us) server_negotiation.cc:407] Connection header received
0812 16:53:04.178980 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:04.178984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:04.179046 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:04.179151 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:04.180016 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.180806 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.182126 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.182649 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.182792 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.187884 (+ 5092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:58353: BlockingRecv error: recv got EOF from 127.8.195.130:58353 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:04 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:04.674018 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:04.663658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:59281)
0812 16:53:04.664586 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:04.664609 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:04.664756 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:04.665210 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:04.665214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:04.665888 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:04.666145 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.666159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.667277 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.667281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.668246 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:04.668258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.669012 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.669016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:04.669214 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.669932 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:04.669952 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:04.673789 (+ 3837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":511,"threads_started":1}
W20260812 16:53:04.674182 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:04.664033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:59281 (local address 127.8.195.190:40249)
0812 16:53:04.664229 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:04.664236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:04.664695 (+ 459us) server_negotiation.cc:407] Connection header received
0812 16:53:04.664899 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:04.664906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:04.665019 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:04.665128 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:04.666324 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.667123 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.668398 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:04.668864 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:04.668978 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:04.673981 (+ 5003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:59281: BlockingRecv error: recv got EOF from 127.8.195.129:59281 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:05.110821 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:05.100431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:37095)
0812 16:53:05.100956 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:05.100978 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:05.101106 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:05.101573 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:05.101578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:05.102101 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:05.102386 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.102399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.103609 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.103615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.104654 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.104668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.105557 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.105564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.105767 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.106504 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:05.106525 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:05.110413 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":281,"threads_started":1}
W20260812 16:53:05.110966 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:05.100546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:37095 (local address 127.8.195.190:40249)
0812 16:53:05.100813 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:05.100819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:05.101064 (+ 245us) server_negotiation.cc:407] Connection header received
0812 16:53:05.101270 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:05.101275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:05.101386 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:05.101491 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:05.102547 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.103433 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.104832 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.105381 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.105535 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.110650 (+ 5115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:37095: BlockingRecv error: recv got EOF from 127.8.195.131:37095 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:05.199889 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:05.189231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:33319)
0812 16:53:05.189938 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:05.189962 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:05.190123 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:05.190615 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:05.190618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:05.191216 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:05.191544 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.191556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.192743 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.192748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.193862 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.193876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.194759 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.194764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.194923 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.195590 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:05.195610 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:05.199621 (+ 4011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":315,"threads_started":1}
W20260812 16:53:05.200134 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:05.189269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:33319 (local address 127.8.195.190:40249)
0812 16:53:05.189454 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:05.189459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:05.190055 (+ 596us) server_negotiation.cc:407] Connection header received
0812 16:53:05.190314 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:05.190337 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:05.190423 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:05.190538 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:05.191732 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.192574 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.194045 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.194577 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.194707 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.199886 (+ 5179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:33319: BlockingRecv error: recv got EOF from 127.8.195.130:33319 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:05 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:05.685825 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:05.675245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:34381)
0812 16:53:05.675902 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:05.675924 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:05.676092 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:05.676566 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:05.676572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:05.677143 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:05.677409 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.677425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.678597 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.678605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.679558 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:05.679570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.680377 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.680383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:05.680564 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.681418 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:05.681438 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:05.685551 (+ 4113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":330,"threads_started":1}
W20260812 16:53:05.686148 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:05.675339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34381 (local address 127.8.195.190:40249)
0812 16:53:05.675636 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:05.675644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:05.676055 (+ 411us) server_negotiation.cc:407] Connection header received
0812 16:53:05.676264 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:05.676269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:05.676388 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:05.676514 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:05.677598 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.678448 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.679710 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:05.680163 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:05.680331 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:05.685839 (+ 5508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:34381: BlockingRecv error: recv got EOF from 127.8.195.129:34381 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:06.123498 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:06.111995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:48767)
0812 16:53:06.112792 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:06.112817 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:06.112998 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:06.113473 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:06.113478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:06.113985 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:06.114185 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.114196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.115514 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.115520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.116587 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.116599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.117511 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.117518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.117707 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.118828 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:06.118849 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:06.123220 (+ 4371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":371,"threads_started":1}
W20260812 16:53:06.123775 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:06.112281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:48767 (local address 127.8.195.190:40249)
0812 16:53:06.112586 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:06.112596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:06.112921 (+ 325us) server_negotiation.cc:407] Connection header received
0812 16:53:06.113146 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:06.113212 (+ 66us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:06.113297 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:06.113421 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:06.114369 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.115351 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.116758 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.117324 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.117469 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.123480 (+ 6011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:48767: BlockingRecv error: recv got EOF from 127.8.195.131:48767 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:06.211009 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:06.201083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:59051)
0812 16:53:06.201676 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:06.201693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:06.201837 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:06.202143 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:06.202146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:06.202567 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:06.202767 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.202777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.204096 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.204101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.205079 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.205093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.205980 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.205990 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.206179 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.206878 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:06.206897 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:06.210748 (+ 3851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":342,"threads_started":1}
W20260812 16:53:06.211293 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:06.201264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:59051 (local address 127.8.195.190:40249)
0812 16:53:06.201516 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:06.201522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:06.201817 (+ 295us) server_negotiation.cc:407] Connection header received
0812 16:53:06.201935 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:06.201939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:06.202006 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:06.202096 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:06.202951 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.203934 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.205275 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.205746 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.205863 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.210995 (+ 5132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:59051: BlockingRecv error: recv got EOF from 127.8.195.130:59051 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:06 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:06.697649 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:06.686929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:36715)
0812 16:53:06.687540 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:06.687557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:06.687676 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:06.688101 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:06.688104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:06.688652 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:06.688825 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.688834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.690029 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.690034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.691096 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:06.691110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.692135 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.692139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:06.692286 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.693017 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:06.693036 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:06.697288 (+ 4252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":322,"threads_started":1}
W20260812 16:53:06.697816 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:06.687023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36715 (local address 127.8.195.190:40249)
0812 16:53:06.687245 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:06.687261 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:06.687641 (+ 380us) server_negotiation.cc:407] Connection header received
0812 16:53:06.687845 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:06.687849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:06.687951 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:06.688068 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:06.688978 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.689866 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.691321 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:06.691909 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:06.692111 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:06.697541 (+ 5430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:36715: BlockingRecv error: recv got EOF from 127.8.195.129:36715 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:07.135075 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:07.124741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:52265)
0812 16:53:07.125434 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:07.125456 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:07.125641 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:07.126023 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:07.126027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:07.126486 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:07.126668 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.126677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.127768 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.127774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.128780 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.128792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.129751 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.129758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.129954 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.130753 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:07.130786 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:07.134818 (+ 4032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":410,"threads_started":1}
W20260812 16:53:07.135306 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:07.124971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:52265 (local address 127.8.195.190:40249)
0812 16:53:07.125267 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:07.125273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:07.125563 (+ 290us) server_negotiation.cc:407] Connection header received
0812 16:53:07.125783 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:07.125788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:07.125869 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:07.125987 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:07.126844 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.127612 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.128946 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.129549 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.129710 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.135041 (+ 5331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:52265: BlockingRecv error: recv got EOF from 127.8.195.131:52265 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:07.222270 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:07.212222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:46393)
0812 16:53:07.212833 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:07.212851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:07.213021 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:07.213451 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:07.213456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:07.213938 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:07.214088 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.214096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.215141 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.215154 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.216142 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.216154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.216907 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.216914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.217108 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.218001 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:07.218020 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:07.221917 (+ 3897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":334,"threads_started":1}
W20260812 16:53:07.222411 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:07.212356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:46393 (local address 127.8.195.190:40249)
0812 16:53:07.212616 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:07.212626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:07.212947 (+ 321us) server_negotiation.cc:407] Connection header received
0812 16:53:07.213157 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:07.213197 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:07.213265 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:07.213384 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:07.214242 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.214989 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.216300 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.216729 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.216914 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.222164 (+ 5250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:46393: BlockingRecv error: recv got EOF from 127.8.195.130:46393 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:07 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:07.709300 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:07.698754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:60871)
0812 16:53:07.699431 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:07.699448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:07.699628 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:07.699997 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:07.700002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:07.700462 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:07.700625 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.700634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.701835 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.701842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.702872 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:07.702884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.703624 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.703630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:07.703812 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.704552 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:07.704573 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:07.708900 (+ 4327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":359,"threads_started":1}
W20260812 16:53:07.709517 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:07.698871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:60871 (local address 127.8.195.190:40249)
0812 16:53:07.699172 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:07.699181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:07.699558 (+ 377us) server_negotiation.cc:407] Connection header received
0812 16:53:07.699776 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:07.699785 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:07.699851 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:07.699965 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:07.700807 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.701682 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.703032 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:07.703450 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:07.703601 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:07.709215 (+ 5614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:60871: BlockingRecv error: recv got EOF from 127.8.195.129:60871 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:08.154083 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:08.136280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:55259)
0812 16:53:08.136829 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:08.136852 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:08.136972 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:08.137384 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:08.137388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:08.137850 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:08.138009 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.138018 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.139011 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.139015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.140010 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.140023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.140726 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.140730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.140858 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.149732 (+ 8874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:08.149759 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:08.153788 (+ 4029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":294,"threads_started":1}
W20260812 16:53:08.154439 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:08.136355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:55259 (local address 127.8.195.190:40249)
0812 16:53:08.136584 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:08.136591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:08.136935 (+ 344us) server_negotiation.cc:407] Connection header received
0812 16:53:08.137072 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:08.137076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:08.137150 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:08.137319 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:08.138154 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.138883 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.140148 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.140583 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.140681 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.154178 (+ 13497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:55259: BlockingRecv error: recv got EOF from 127.8.195.131:55259 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:08.233577 27489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:08.223521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:51307)
0812 16:53:08.224245 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:08.224275 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:08.224446 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:08.224836 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:08.224842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:08.225364 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:08.225529 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.225539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.226614 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.226621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.227617 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.227630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.228398 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.228404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.228583 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.229293 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:08.229312 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:08.233065 (+ 3753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":347,"threads_started":1}
W20260812 16:53:08.233650 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:08.223699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:51307 (local address 127.8.195.190:40249)
0812 16:53:08.223974 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:08.223980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:08.224369 (+ 389us) server_negotiation.cc:407] Connection header received
0812 16:53:08.224599 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:08.224615 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:08.224689 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:08.224804 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:08.225695 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.226464 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.227778 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.228235 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.228380 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.233350 (+ 4970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:51307: BlockingRecv error: recv got EOF from 127.8.195.130:51307 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:08 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:08.720484 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:08.710480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:36437)
0812 16:53:08.711286 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:08.711304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:08.711459 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:08.711805 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:08.711809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:08.712244 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:08.712435 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.712445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.713605 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.713609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.714560 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:08.714572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.715406 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.715411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:08.715540 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.716297 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:08.716317 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:08.720227 (+ 3910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":575,"threads_started":1}
W20260812 16:53:08.720721 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:08.710823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36437 (local address 127.8.195.190:40249)
0812 16:53:08.711074 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:08.711080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:08.711407 (+ 327us) server_negotiation.cc:407] Connection header received
0812 16:53:08.711619 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:08.711623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:08.711688 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:08.711770 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:08.712612 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.713473 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.714734 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:08.715222 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:08.715332 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:08.720458 (+ 5126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:36437: BlockingRecv error: recv got EOF from 127.8.195.129:36437 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:09.165728 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:09.155299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:48063)
0812 16:53:09.156011 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:09.156061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:09.156257 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:09.156672 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:09.156678 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:09.157219 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:09.157410 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.157420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.158526 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.158536 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.159543 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.159556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.160380 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.160387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.160573 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.161360 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:09.161380 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:09.165455 (+ 4075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":442,"threads_started":1}
W20260812 16:53:09.166172 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:09.155424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:48063 (local address 127.8.195.190:40249)
0812 16:53:09.155749 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:09.155758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:09.156176 (+ 418us) server_negotiation.cc:407] Connection header received
0812 16:53:09.156435 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:09.156440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:09.156507 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:09.156627 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:09.157578 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.158355 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.159709 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.160167 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.160332 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.165820 (+ 5488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:48063: BlockingRecv error: recv got EOF from 127.8.195.131:48063 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:09.244915 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:09.234739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:47453)
0812 16:53:09.235324 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:09.235342 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:09.235525 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:09.236055 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:09.236060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:09.236607 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:09.236808 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.236818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.237991 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.237995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.238971 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.238985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.239822 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.239827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.239985 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.240700 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:09.240718 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:09.244651 (+ 3933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":352,"threads_started":1}
W20260812 16:53:09.245265 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:09.234889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:47453 (local address 127.8.195.190:40249)
0812 16:53:09.235172 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:09.235180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:09.235467 (+ 287us) server_negotiation.cc:407] Connection header received
0812 16:53:09.235777 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:09.235784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:09.235867 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:09.235997 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:09.236972 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.237833 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.239136 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.239611 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.239749 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.244967 (+ 5218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:47453: BlockingRecv error: recv got EOF from 127.8.195.130:47453 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:09 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:09.731814 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:09.721648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:59551)
0812 16:53:09.722309 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:09.722327 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:09.722451 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:09.722811 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:09.722815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:09.723250 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:09.723447 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.723457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.724738 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.724742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.725781 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:09.725794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.726655 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.726661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:09.726859 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.727578 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:09.727597 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:09.731428 (+ 3831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":370,"threads_started":1}
W20260812 16:53:09.731971 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:09.721779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:59551 (local address 127.8.195.190:40249)
0812 16:53:09.721979 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:09.721983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:09.722408 (+ 425us) server_negotiation.cc:407] Connection header received
0812 16:53:09.722559 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:09.722563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:09.722629 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:09.722724 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:09.723625 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.724589 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.725929 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:09.726444 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:09.726608 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:09.731662 (+ 5054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:59551: BlockingRecv error: recv got EOF from 127.8.195.129:59551 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:10.177093 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:10.166995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:59043)
0812 16:53:10.167586 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:10.167606 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:10.167780 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:10.168141 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:10.168145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:10.168608 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:10.168832 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.168843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.169994 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.170000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.170978 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.170992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.171778 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.171784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.171999 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.172760 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:10.172780 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:10.176744 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":335,"threads_started":1}
W20260812 16:53:10.177261 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:10.167097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:59043 (local address 127.8.195.190:40249)
0812 16:53:10.167437 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:10.167444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:10.167704 (+ 260us) server_negotiation.cc:407] Connection header received
0812 16:53:10.167928 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:10.167932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:10.167996 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:10.168103 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:10.169003 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.169828 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.171147 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.171602 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.171758 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.176958 (+ 5200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:59043: BlockingRecv error: recv got EOF from 127.8.195.131:59043 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:10.256772 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:10.246002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:60827)
0812 16:53:10.246714 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:10.246731 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:10.246867 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:10.247244 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:10.247249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:10.247718 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:10.247922 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.247933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.249161 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.249263 (+ 102us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.250255 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.250271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.251184 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.251190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.251373 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.252126 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:10.252147 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:10.256515 (+ 4368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":428,"threads_started":1}
W20260812 16:53:10.257045 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:10.246162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:60827 (local address 127.8.195.190:40249)
0812 16:53:10.246400 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:10.246405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:10.246844 (+ 439us) server_negotiation.cc:407] Connection header received
0812 16:53:10.247012 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:10.247016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:10.247089 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:10.247191 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:10.248093 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.248973 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.250429 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.250964 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.251079 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.256762 (+ 5683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:60827: BlockingRecv error: recv got EOF from 127.8.195.130:60827 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:10 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:10.742821 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:10.733110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:48879)
0812 16:53:10.733675 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:10.733692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:10.733866 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:10.734285 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:10.734289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:10.734703 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:10.734898 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.734908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.736007 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.736011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.737048 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:10.737062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.737860 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.737865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:10.737998 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.738661 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:10.738680 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:10.742560 (+ 3880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":355,"threads_started":1}
W20260812 16:53:10.743177 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:10.733241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:48879 (local address 127.8.195.190:40249)
0812 16:53:10.733462 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:10.733471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:10.733787 (+ 316us) server_negotiation.cc:407] Connection header received
0812 16:53:10.734000 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:10.734005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:10.734105 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:10.734214 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:10.735048 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.735840 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.737242 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:10.737713 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:10.737824 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:10.742826 (+ 5002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:48879: BlockingRecv error: recv got EOF from 127.8.195.129:48879 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:11.189494 27502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:11.178508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:34729)
0812 16:53:11.179160 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:11.179185 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:11.179344 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:11.179704 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:11.179709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:11.180276 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:11.180494 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.180505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.181875 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.181884 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.183166 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.183183 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.184195 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.184200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.184419 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.185244 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:11.185272 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:11.189151 (+ 3879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":364,"threads_started":1}
W20260812 16:53:11.189750 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:11.178589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:34729 (local address 127.8.195.190:40249)
0812 16:53:11.178900 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:11.178907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:11.179301 (+ 394us) server_negotiation.cc:407] Connection header received
0812 16:53:11.179463 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:11.179467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:11.179539 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:11.179633 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:11.180704 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.181638 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.183347 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.184028 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.184177 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.189478 (+ 5301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:34729: BlockingRecv error: recv got EOF from 127.8.195.131:34729 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:11.268438 27503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:11.257994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:49583)
0812 16:53:11.258590 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:11.258613 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:11.258799 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:11.259450 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:11.259455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:11.259996 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:11.260227 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.260240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.261576 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.261582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.262501 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.262512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.263302 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.263309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.263491 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.264203 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:11.264222 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:11.268070 (+ 3848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":315,"threads_started":1}
W20260812 16:53:11.268584 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:11.258319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:49583 (local address 127.8.195.190:40249)
0812 16:53:11.258579 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:11.258586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:11.258718 (+ 132us) server_negotiation.cc:407] Connection header received
0812 16:53:11.258950 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:11.258954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:11.259019 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:11.259137 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:11.260405 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.261429 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.262657 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.263123 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.263269 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.268319 (+ 5050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:49583: BlockingRecv error: recv got EOF from 127.8.195.130:49583 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:11 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:11.755129 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:11.744276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:49377)
0812 16:53:11.744864 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:11.744881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:11.745005 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:11.745521 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:11.745526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:11.746031 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:11.746240 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.746251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.747480 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.747484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.748548 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:11.748562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.749342 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.749347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:11.749615 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.750381 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:11.750401 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:11.754853 (+ 4452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":318,"threads_started":1}
W20260812 16:53:11.755335 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:11.744376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49377 (local address 127.8.195.190:40249)
0812 16:53:11.744568 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:11.744572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:11.744994 (+ 422us) server_negotiation.cc:407] Connection header received
0812 16:53:11.745213 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:11.745237 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:11.745314 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:11.745410 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:11.746430 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.747303 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.748725 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:11.749157 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:11.749308 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:11.755079 (+ 5771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:49377: BlockingRecv error: recv got EOF from 127.8.195.129:49377 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.131@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:12.202003 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.190688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.131:38063)
0812 16:53:12.191356 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:12.191379 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:12.191561 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:12.191946 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:12.191951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:12.192492 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:12.192667 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.192677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.193952 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.193965 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.194993 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.195005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.195856 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.195863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.196074 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.196864 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:12.196884 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:12.201622 (+ 4738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":327,"threads_started":1}
W20260812 16:53:12.202215 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:12.190746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:38063 (local address 127.8.195.190:40249)
0812 16:53:12.191153 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.191162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.191477 (+ 315us) server_negotiation.cc:407] Connection header received
0812 16:53:12.191699 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.191705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.191793 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.191910 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.192860 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.193783 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.195165 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.195659 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.195835 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.201952 (+ 6117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.131:38063: BlockingRecv error: recv got EOF from 127.8.195.131:38063 (error 108)
Metrics: {"server-negotiator.queue_time_us":223}
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.130@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:12.280481 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.269796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.130:33047)
0812 16:53:12.270420 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:12.270438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:12.270562 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:12.270990 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:12.270995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:12.271432 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:12.271649 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.271659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.272796 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.272801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.273861 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.273877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.274748 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.274753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.274902 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.275706 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:12.275726 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:12.280065 (+ 4339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":322,"threads_started":1}
W20260812 16:53:12.280635 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:12.270021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:33047 (local address 127.8.195.190:40249)
0812 16:53:12.270230 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.270235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.270525 (+ 290us) server_negotiation.cc:407] Connection header received
0812 16:53:12.270671 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.270675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.270779 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.270881 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.271836 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.272609 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.274036 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.274555 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.274675 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.280316 (+ 5641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.130:33047: BlockingRecv error: recv got EOF from 127.8.195.130:33047 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 16:53:12 dist-test-slave-z8x9 krb5kdc[25574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.195.129@KRBTEST.COM for oryx/127.8.195.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 16:53:12.766860 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.756454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:40249 (local address 127.8.195.129:41029)
0812 16:53:12.757144 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:12.757162 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:12.757388 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:12.757818 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:12.757822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:12.758346 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:12.758613 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.758624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.759793 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.759800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.760796 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:12.760809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.761626 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.761631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:12.761813 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.762540 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:12.762560 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:12.766569 (+ 4009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.195.190:40249: Server oryx/127.8.195.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":410,"threads_started":1}
W20260812 16:53:12.767222 27437 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:12.756615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:41029 (local address 127.8.195.190:40249)
0812 16:53:12.756811 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.756816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.757356 (+ 540us) server_negotiation.cc:407] Connection header received
0812 16:53:12.757516 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.757521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.757614 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.757733 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.758788 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.759631 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.760965 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.761461 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.761601 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.766894 (+ 5293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.195.129:41029: BlockingRecv error: recv got EOF from 127.8.195.129:41029 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:53:12.847056 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26659
I20260812 16:53:12.862191 27516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.848694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.8.195.129:35273)
0812 16:53:12.849496 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.849503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.849535 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:53:12.849602 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.849609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.849858 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.850050 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.851612 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.852754 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.854130 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.854859 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.855000 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.858263 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:12.858275 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:12.858292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:12.858347 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:12.861589 (+ 3242us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 16:53:12.861946 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55414: Request ticket server kudu/127.8.195.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":451,"threads_started":1}
W20260812 16:53:12.862291 27516 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55414: Request ticket server kudu/127.8.195.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 16:53:12.862581 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26659: Not authorized: Client connection negotiation failed: client connection to 127.8.195.129:35273: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 16:53:12.862845 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 26659
I20260812 16:53:12.905237 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26952
I20260812 16:53:12.918941 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.906476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.8.195.130:35139)
0812 16:53:12.907086 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.907093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.907130 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:53:12.907202 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.907209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.907446 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.907675 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.909089 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.910235 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.911541 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.912124 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.912277 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.915100 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:12.915110 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:12.915125 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:12.915172 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:12.918159 (+ 2987us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 16:53:12.918583 (+ 424us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57166: Request ticket server kudu/127.8.195.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":302,"threads_started":1}
W20260812 16:53:12.919081 27517 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57166: Request ticket server kudu/127.8.195.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 16:53:12.919075 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26952: Not authorized: Client connection negotiation failed: client connection to 127.8.195.130:35139: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 16:53:12.919241 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 26952
I20260812 16:53:12.954834 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 27187
I20260812 16:53:12.969050 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:12.956100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.8.195.131:43595)
0812 16:53:12.956883 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:12.956893 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:12.956926 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:53:12.957008 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:12.957020 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:12.957411 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:12.957650 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:12.959081 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.960154 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.961501 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:12.962143 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:12.962253 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:12.965042 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:12.965051 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:12.965067 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:12.965121 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:12.968450 (+ 3329us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 16:53:12.968793 (+ 343us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40468: Request ticket server kudu/127.8.195.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":341,"threads_started":1}
W20260812 16:53:12.969228 27519 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40468: Request ticket server kudu/127.8.195.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 16:53:12.969301 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 27187: Not authorized: Client connection negotiation failed: client connection to 127.8.195.131:43595: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.195.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 16:53:12.969507 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 27187
I20260812 16:53:13.003646 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 26525
I20260812 16:53:13.005260 26645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:13.132962 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 26525
2026-08-12T16:53:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22562 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T16:53:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:53:13Z Disabled control of system clock
I20260812 16:53:13.243361 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39243
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:35557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:39243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 16:53:13.606454 27525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:13.606801 27525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:13.606945 27525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:13.614481 27525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:53:13.614615 27525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:13.614686 27525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:13.614777 27525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:53:13.614861 27525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:53:13.624660 27525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:39243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:39243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:13.626179 27525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:13.628141 27525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:13.641198 27533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:13.641096 27534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:13.641311 27538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:13.642526 27525 server_base.cc:1034] running on GCE node
I20260812 16:53:13.643333 27525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:13.644786 27525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:13.645992 27525 hybrid_clock.cc:648] HybridClock initialized: now 1786553593645957 us; error 27 us; skew 500 ppm
I20260812 16:53:13.646351 27525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:13.652326 27525 webserver.cc:460] Webserver started at http://127.8.195.190:36425/ using document root <none> and password file <none>
I20260812 16:53:13.653162 27525 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:13.653349 27525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:13.653666 27525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:13.656824 27525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "483d997edaef4e5f87ba043b8122e4ae"
format_stamp: "Formatted at 2026-08-12 16:53:13 on dist-test-slave-z8x9"
server_key: "510fb12138d46d72b792a136b7c8d504"
server_key_iv: "6ac140d592c5ac28ad04a644c7c039f0"
server_key_version: "encryptionkey@0"
I20260812 16:53:13.657982 27525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "483d997edaef4e5f87ba043b8122e4ae"
format_stamp: "Formatted at 2026-08-12 16:53:13 on dist-test-slave-z8x9"
server_key: "510fb12138d46d72b792a136b7c8d504"
server_key_iv: "6ac140d592c5ac28ad04a644c7c039f0"
server_key_version: "encryptionkey@0"
I20260812 16:53:13.664482 27525 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 16:53:13.668578 27547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:13.670308 27525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:53:13.670490 27525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "483d997edaef4e5f87ba043b8122e4ae"
format_stamp: "Formatted at 2026-08-12 16:53:13 on dist-test-slave-z8x9"
server_key: "510fb12138d46d72b792a136b7c8d504"
server_key_iv: "6ac140d592c5ac28ad04a644c7c039f0"
server_key_version: "encryptionkey@0"
I20260812 16:53:13.670701 27525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:13.687661 27525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:13.688601 27525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:13.688881 27525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:13.709875 27525 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:39243
I20260812 16:53:13.709898 27638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:39243 every 8 connection(s)
I20260812 16:53:13.711404 27525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:53:13.715737 27641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:13.715814 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 27525
I20260812 16:53:13.716015 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:53:13.716698 8974 external_mini_cluster.cc:1428] Setting key 7b259b0b12fe47589db88b1c9de2ff2e
I20260812 16:53:13.731750 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae: Bootstrap starting.
I20260812 16:53:13.735402 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:13.718990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.8.195.190:39243)
0812 16:53:13.720243 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:13.720261 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:13.720317 (+ 56us) server_negotiation.cc:407] Connection header received
0812 16:53:13.723199 (+ 2882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:13.723239 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:13.724016 (+ 777us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:13.724570 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:13.725584 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:13.727576 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:13.729394 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:13.730553 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:13.730778 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:13.732265 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:13.732278 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:13.732301 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:13.733096 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:13.733248 (+ 152us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:13.733294 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:13.733423 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:13.733808 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 16:53:13.734136 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":402,"threads_started":1}
I20260812 16:53:13.736114 27641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:13.737422 27641 log.cc:824] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:13.740713 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae: No bootstrap required, opened a new log
I20260812 16:53:13.749321 27641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } }
I20260812 16:53:13.749902 27641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:13.750036 27641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483d997edaef4e5f87ba043b8122e4ae, State: Initialized, Role: FOLLOWER
I20260812 16:53:13.750771 27641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } }
I20260812 16:53:13.751044 27641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:53:13.751224 27641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:53:13.751415 27641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:13.754119 27641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } }
I20260812 16:53:13.754794 27641 leader_election.cc:302] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 483d997edaef4e5f87ba043b8122e4ae; no voters:
I20260812 16:53:13.755542 27641 leader_election.cc:288] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:53:13.755801 27653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:53:13.756646 27653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [term 1 LEADER]: Becoming Leader. State: Replica: 483d997edaef4e5f87ba043b8122e4ae, State: Running, Role: LEADER
I20260812 16:53:13.757436 27653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } }
I20260812 16:53:13.757990 27641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:53:13.760998 27655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 483d997edaef4e5f87ba043b8122e4ae. Latest consensus state: current_term: 1 leader_uuid: "483d997edaef4e5f87ba043b8122e4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } } }
I20260812 16:53:13.761575 27655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:13.760993 27654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "483d997edaef4e5f87ba043b8122e4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "483d997edaef4e5f87ba043b8122e4ae" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 39243 } } }
I20260812 16:53:13.761837 27654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:13.764142 27666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:53:13.769941 27666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:53:13.781038 27666 catalog_manager.cc:1347] Generated new cluster ID: 4a1d0c35b6e34029b13f851032934643
I20260812 16:53:13.781245 27666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:53:13.794191 27666 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:53:13.796329 27666 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:53:13.815833 27666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 483d997edaef4e5f87ba043b8122e4ae: Generated new TSK 0
I20260812 16:53:13.816805 27666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:53:13.822494 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--builtin_ntp_servers=127.8.195.148:35557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 16:53:14.189913 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:14.190246 27693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:14.190335 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:14.197830 27693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:14.197947 27693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:14.198053 27693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:53:14.207399 27693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:14.208755 27693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:14.210585 27693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:14.224994 27702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:14.224941 27700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:14.225041 27706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:14.226487 27693 server_base.cc:1034] running on GCE node
I20260812 16:53:14.227329 27693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:14.228417 27693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:14.229660 27693 hybrid_clock.cc:648] HybridClock initialized: now 1786553594229606 us; error 52 us; skew 500 ppm
I20260812 16:53:14.230021 27693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:14.232438 27693 webserver.cc:460] Webserver started at http://127.8.195.129:40203/ using document root <none> and password file <none>
I20260812 16:53:14.233301 27693 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:14.233445 27693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:14.233812 27693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:14.237269 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "312b7b064163498f8f5959d597c36e3d"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "1e078a73db1364ea1094273685269389"
server_key_iv: "fc76aa758d86ec43576ba42688324565"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.238163 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "312b7b064163498f8f5959d597c36e3d"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "1e078a73db1364ea1094273685269389"
server_key_iv: "fc76aa758d86ec43576ba42688324565"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.243904 27693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:53:14.247596 27718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.249018 27693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 16:53:14.249275 27693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "312b7b064163498f8f5959d597c36e3d"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "1e078a73db1364ea1094273685269389"
server_key_iv: "fc76aa758d86ec43576ba42688324565"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.249526 27693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:14.266290 27693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:14.267231 27693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:14.267571 27693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:14.268656 27693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:14.270351 27693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:14.270489 27693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.270627 27693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:14.270725 27693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.306835 27693 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:37371
I20260812 16:53:14.306911 27916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:37371 every 8 connection(s)
I20260812 16:53:14.308862 27693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:53:14.315157 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 27693
I20260812 16:53:14.315375 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:53:14.316216 8974 external_mini_cluster.cc:1428] Setting key 342da059f1394ec03abe0d1caf0cb9a3
I20260812 16:53:14.320057 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--builtin_ntp_servers=127.8.195.148:35557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:53:14.329617 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:14.312776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39243 (local address 127.8.195.129:60819)
0812 16:53:14.314277 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:14.314356 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:14.316905 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:14.319126 (+ 2221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:14.319136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:14.319243 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:14.319897 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:14.319917 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.321657 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.321663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:14.324236 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:14.324257 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.325557 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.325566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:14.325948 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:14.326814 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:14.326820 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:14.326892 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:14.327009 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:14.327500 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:14.327512 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:14.327523 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:53:14.327700 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 16:53:14.328164 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"spinlock_wait_cycles":18688,"thread_start_us":558,"threads_started":1}
I20260812 16:53:14.330433 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:14.313313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:60819 (local address 127.8.195.190:39243)
0812 16:53:14.313772 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:14.313779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:14.314522 (+ 743us) server_negotiation.cc:407] Connection header received
0812 16:53:14.318492 (+ 3970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:14.318497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:14.318594 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:14.318705 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:14.320097 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.321414 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.324449 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.325112 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.325260 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:14.327268 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:14.327277 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:14.327283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:14.327311 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:14.327363 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:14.327381 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:14.327447 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:14.330062 (+ 2615us) server_negotiation.cc:299] Negotiation successful
0812 16:53:14.330170 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":898,"threads_started":1}
I20260812 16:53:14.332456 27917 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39243
I20260812 16:53:14.332844 27917 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:14.334154 27917 heartbeater.cc:511] Master 127.8.195.190:39243 requested a full tablet report, sending...
I20260812 16:53:14.337286 27576 ts_manager.cc:194] Registered new tserver with Master: 312b7b064163498f8f5959d597c36e3d (127.8.195.129:37371)
I20260812 16:53:14.339777 27576 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.129:60819
W20260812 16:53:14.692335 27927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:14.692685 27927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:14.692837 27927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:14.700477 27927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:14.700601 27927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:14.700788 27927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:53:14.711437 27927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:14.713061 27927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:14.715397 27927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:14.731209 27935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:14.731177 27937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:14.731658 27934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:14.732571 27927 server_base.cc:1034] running on GCE node
I20260812 16:53:14.733359 27927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:14.734391 27927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:14.735643 27927 hybrid_clock.cc:648] HybridClock initialized: now 1786553594735580 us; error 55 us; skew 500 ppm
I20260812 16:53:14.735992 27927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:14.738610 27927 webserver.cc:460] Webserver started at http://127.8.195.130:42661/ using document root <none> and password file <none>
I20260812 16:53:14.739339 27927 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:14.739437 27927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:14.739723 27927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:14.742894 27927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "b933da39fe5f41bd8f3cf1386e8b7472"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "853172a56fa4431ace71bd320449e4ef"
server_key_iv: "4ca537c1c36c8a8d379f18b790b512a7"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.743672 27927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "b933da39fe5f41bd8f3cf1386e8b7472"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "853172a56fa4431ace71bd320449e4ef"
server_key_iv: "4ca537c1c36c8a8d379f18b790b512a7"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.748953 27927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 16:53:14.752717 27948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.754350 27927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:53:14.754608 27927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b933da39fe5f41bd8f3cf1386e8b7472"
format_stamp: "Formatted at 2026-08-12 16:53:14 on dist-test-slave-z8x9"
server_key: "853172a56fa4431ace71bd320449e4ef"
server_key_iv: "4ca537c1c36c8a8d379f18b790b512a7"
server_key_version: "encryptionkey@0"
I20260812 16:53:14.754858 27927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:14.770077 27927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:14.771032 27927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:14.771378 27927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:14.772428 27927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:14.774068 27927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:14.774165 27927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.774359 27927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:14.774446 27927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:14.810204 27927 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:35673
I20260812 16:53:14.810333 28159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:35673 every 8 connection(s)
I20260812 16:53:14.811858 27927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:53:14.820061 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 27927
I20260812 16:53:14.820243 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:53:14.820729 8974 external_mini_cluster.cc:1428] Setting key af1b588f458e6930e45b97182e63cec5
I20260812 16:53:14.823976 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--builtin_ntp_servers=127.8.195.148:35557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 16:53:14.826157 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:14.814750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:52635 (local address 127.8.195.190:39243)
0812 16:53:14.815006 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:14.815013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:14.816551 (+ 1538us) server_negotiation.cc:407] Connection header received
0812 16:53:14.818366 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:14.818371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:14.818446 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:14.818544 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:14.819875 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.820917 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.822737 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.823339 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.823479 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:14.825095 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:14.825105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:14.825113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:14.825142 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:14.825216 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:14.825238 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:14.825344 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:14.825586 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:53:14.825717 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 16:53:14.827121 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:14.815302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39243 (local address 127.8.195.130:52635)
0812 16:53:14.816372 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:14.816411 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:14.818088 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:14.818770 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:14.818779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:14.818874 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:14.819609 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:14.819642 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.821117 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.821123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:14.822553 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:14.822568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:14.823582 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:14.823588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:14.823830 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:14.824709 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:14.824712 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:14.824769 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:14.824865 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:14.825347 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:14.825356 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:14.825363 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:53:14.825538 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 16:53:14.825902 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":329,"threads_started":1}
I20260812 16:53:14.828847 28163 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39243
I20260812 16:53:14.829245 28163 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:14.830304 28163 heartbeater.cc:511] Master 127.8.195.190:39243 requested a full tablet report, sending...
I20260812 16:53:14.832262 27576 ts_manager.cc:194] Registered new tserver with Master: b933da39fe5f41bd8f3cf1386e8b7472 (127.8.195.130:35673)
I20260812 16:53:14.833256 27576 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.130:52635
W20260812 16:53:15.191345 28170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:15.191691 28170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:15.191839 28170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:15.199867 28170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:15.200006 28170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:15.200188 28170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:53:15.209944 28170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:39243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:15.211380 28170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:15.213306 28170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:15.226933 28181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:15.227901 28182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:15.228741 28184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:15.229688 28170 server_base.cc:1034] running on GCE node
I20260812 16:53:15.230474 28170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:15.231645 28170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:15.232944 28170 hybrid_clock.cc:648] HybridClock initialized: now 1786553595232881 us; error 63 us; skew 500 ppm
I20260812 16:53:15.233472 28170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:15.236202 28170 webserver.cc:460] Webserver started at http://127.8.195.131:44217/ using document root <none> and password file <none>
I20260812 16:53:15.237268 28170 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:15.237376 28170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:15.237746 28170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:15.240983 28170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "7a18c448eb384327882d3faf335583da"
format_stamp: "Formatted at 2026-08-12 16:53:15 on dist-test-slave-z8x9"
server_key: "7a1514fe469506f6b59b52c00e9534e7"
server_key_iv: "665cee8d117600da53ea14d87d075acc"
server_key_version: "encryptionkey@0"
I20260812 16:53:15.241919 28170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "7a18c448eb384327882d3faf335583da"
format_stamp: "Formatted at 2026-08-12 16:53:15 on dist-test-slave-z8x9"
server_key: "7a1514fe469506f6b59b52c00e9534e7"
server_key_iv: "665cee8d117600da53ea14d87d075acc"
server_key_version: "encryptionkey@0"
I20260812 16:53:15.247891 28170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 16:53:15.251992 28193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:15.253636 28170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:53:15.253857 28170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "7a18c448eb384327882d3faf335583da"
format_stamp: "Formatted at 2026-08-12 16:53:15 on dist-test-slave-z8x9"
server_key: "7a1514fe469506f6b59b52c00e9534e7"
server_key_iv: "665cee8d117600da53ea14d87d075acc"
server_key_version: "encryptionkey@0"
I20260812 16:53:15.254166 28170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:15.271675 28170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:15.272667 28170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:15.273008 28170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:15.274227 28170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:15.275914 28170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:15.276046 28170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:15.276185 28170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:15.276275 28170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:15.311568 28170 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:39023
I20260812 16:53:15.311590 28405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:39023 every 8 connection(s)
I20260812 16:53:15.312908 28170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:53:15.318357 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 28170
I20260812 16:53:15.318552 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:53:15.319157 8974 external_mini_cluster.cc:1428] Setting key 503f3ed46cbf2cdc9fb178ea24bf1ecd
I20260812 16:53:15.327759 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:15.315858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:37713 (local address 127.8.195.190:39243)
0812 16:53:15.316065 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:15.316070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:15.317772 (+ 1702us) server_negotiation.cc:407] Connection header received
0812 16:53:15.320489 (+ 2717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:15.320494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:15.320585 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:15.320703 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:15.322019 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.322814 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.324566 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.324985 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.325135 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:15.326769 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:15.326775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:15.326778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:15.326799 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:15.326833 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:15.326846 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:15.326950 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:15.327228 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 16:53:15.327346 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 16:53:15.329051 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:15.316388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:39243 (local address 127.8.195.131:37713)
0812 16:53:15.317579 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:15.317621 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:15.320026 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:15.320982 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:15.320991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:15.321093 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:15.321821 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:15.321841 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.322983 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.322987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:15.324372 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:15.324388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.325273 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.325281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:15.325541 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:15.326372 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:15.326375 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:15.326432 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:15.326527 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:15.326952 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:15.326973 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:15.326988 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 16:53:15.327186 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 16:53:15.327693 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":391,"threads_started":1}
I20260812 16:53:15.330785 28406 heartbeater.cc:347] Connected to a master server at 127.8.195.190:39243
I20260812 16:53:15.331156 28406 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:15.332048 28406 heartbeater.cc:511] Master 127.8.195.190:39243 requested a full tablet report, sending...
I20260812 16:53:15.333896 27576 ts_manager.cc:194] Registered new tserver with Master: 7a18c448eb384327882d3faf335583da (127.8.195.131:39023)
I20260812 16:53:15.334919 27576 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.131:37713
I20260812 16:53:15.335563 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:53:15.342944 27923 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:15.341286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.8.195.190:39243)
0812 16:53:15.341470 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:15.341475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:15.341997 (+ 522us) server_negotiation.cc:407] Connection header received
0812 16:53:15.342149 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:15.342153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:15.342224 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:15.342345 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:15.342714 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49810: BlockingRecv error: recv got EOF from 127.0.0.1:49810 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:53:15.343432 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 27693
I20260812 16:53:15.347656 27917 heartbeater.cc:503] Master 127.8.195.190:39243 was elected leader, sending a full tablet report...
I20260812 16:53:15.353327 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:15.344552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.8.195.129:37371)
0812 16:53:15.345220 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:15.345230 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:15.345263 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:53:15.345337 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:15.345344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:15.345573 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:15.345701 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:15.346127 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.347180 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.348621 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.349600 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.349726 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:15.351469 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:15.351482 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:15.351501 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:15.352310 (+ 809us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:15.352391 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:15.352421 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:15.352562 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:15.352706 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 16:53:15.352836 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":312,"threads_started":1}
I20260812 16:53:15.354686 27907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:15.441131 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 27693
I20260812 16:53:15.466578 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 27927
I20260812 16:53:15.476014 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:15.467600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.8.195.130:35673)
0812 16:53:15.468202 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:15.468209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:15.468239 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:53:15.468302 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:15.468318 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:15.468551 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:15.468690 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:15.469294 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.470352 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.471713 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.472394 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.472525 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:15.474149 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:15.474169 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:15.474212 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:15.474968 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:15.475028 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:15.475054 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:15.475179 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:15.475463 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 16:53:15.475595 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":322,"threads_started":1}
I20260812 16:53:15.477569 28152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:15.562935 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 27927
I20260812 16:53:15.591451 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 28170
I20260812 16:53:15.600886 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:15.592842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.8.195.131:39023)
0812 16:53:15.593687 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:15.593694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:15.593731 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:53:15.593798 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:15.593806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:15.594052 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:15.594196 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:15.594780 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.595885 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.597282 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:15.597941 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:15.598051 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:15.599219 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:15.599228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:15.599243 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:15.599891 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:15.599945 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:15.599974 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:15.600072 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:15.600254 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 16:53:15.600406 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":518,"threads_started":1}
I20260812 16:53:15.602331 28398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:15.688890 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 28170
I20260812 16:53:15.712709 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 27525
I20260812 16:53:15.714145 27627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:15.820775 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 27525
2026-08-12T16:53:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2679 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T16:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:53:15Z Disabled control of system clock
I20260812 16:53:15.925390 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41023
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:41023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:53:16.294339 28442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:16.294673 28442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:16.294848 28442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:16.302412 28442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:53:16.302538 28442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:16.302620 28442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:16.302708 28442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:53:16.302774 28442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:53:16.312404 28442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:41023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:16.313815 28442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:16.315644 28442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:16.328243 28453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:16.328336 28450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:16.328286 28449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:16.329268 28442 server_base.cc:1034] running on GCE node
I20260812 16:53:16.330289 28442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:16.331791 28442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:16.333081 28442 hybrid_clock.cc:648] HybridClock initialized: now 1786553596333033 us; error 64 us; skew 500 ppm
I20260812 16:53:16.333514 28442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:16.336223 28442 webserver.cc:460] Webserver started at http://127.8.195.190:35315/ using document root <none> and password file <none>
I20260812 16:53:16.337080 28442 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:16.337265 28442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:16.337670 28442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:16.340981 28442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "b528c98b345d48baa04127ad1f802d7d"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "f07965cfea22be990e9752a832f44106"
server_key_iv: "086739d1d1bfa804165624827c25de6d"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.341879 28442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "b528c98b345d48baa04127ad1f802d7d"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "f07965cfea22be990e9752a832f44106"
server_key_iv: "086739d1d1bfa804165624827c25de6d"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.347692 28442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:53:16.351605 28461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:16.353615 28442 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:53:16.353855 28442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "b528c98b345d48baa04127ad1f802d7d"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "f07965cfea22be990e9752a832f44106"
server_key_iv: "086739d1d1bfa804165624827c25de6d"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.354244 28442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:16.361846 28442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:16.362766 28442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:16.363062 28442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:16.384065 28442 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:41023
I20260812 16:53:16.384111 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:41023 every 8 connection(s)
I20260812 16:53:16.385730 28442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:53:16.388855 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 28442
I20260812 16:53:16.389093 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:53:16.389834 8974 external_mini_cluster.cc:1428] Setting key da534fe5c00894b324bd788218de6b2c
I20260812 16:53:16.390443 28562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:16.400159 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:16.392336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.8.195.190:41023)
0812 16:53:16.393613 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:16.393650 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:16.393711 (+ 61us) server_negotiation.cc:407] Connection header received
0812 16:53:16.395907 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:16.395934 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:16.396445 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:16.396921 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:16.398343 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:16.398350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:16.398367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:16.398389 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:16.398454 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:16.398473 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:16.398750 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:16.398899 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 16:53:16.399171 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":502,"threads_started":1}
I20260812 16:53:16.405277 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d: Bootstrap starting.
I20260812 16:53:16.408648 28562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:16.409804 28562 log.cc:824] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:16.413007 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d: No bootstrap required, opened a new log
I20260812 16:53:16.420776 28562 raft_consensus.cc:348] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } }
I20260812 16:53:16.421277 28562 raft_consensus.cc:374] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:16.421411 28562 raft_consensus.cc:729] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b528c98b345d48baa04127ad1f802d7d, State: Initialized, Role: FOLLOWER
I20260812 16:53:16.422094 28562 consensus_queue.cc:260] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } }
I20260812 16:53:16.422367 28562 raft_consensus.cc:388] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:53:16.422534 28562 raft_consensus.cc:482] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:53:16.422715 28562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:16.425119 28562 raft_consensus.cc:504] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } }
I20260812 16:53:16.425912 28562 leader_election.cc:302] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b528c98b345d48baa04127ad1f802d7d; no voters:
I20260812 16:53:16.426508 28562 leader_election.cc:288] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:53:16.426673 28570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:53:16.427426 28570 raft_consensus.cc:686] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [term 1 LEADER]: Becoming Leader. State: Replica: b528c98b345d48baa04127ad1f802d7d, State: Running, Role: LEADER
I20260812 16:53:16.427990 28570 consensus_queue.cc:237] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } }
I20260812 16:53:16.428746 28562 sys_catalog.cc:564] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:53:16.431128 28578 sys_catalog.cc:455] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b528c98b345d48baa04127ad1f802d7d. Latest consensus state: current_term: 1 leader_uuid: "b528c98b345d48baa04127ad1f802d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } } }
I20260812 16:53:16.431519 28578 sys_catalog.cc:458] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:16.431233 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b528c98b345d48baa04127ad1f802d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b528c98b345d48baa04127ad1f802d7d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41023 } } }
I20260812 16:53:16.431943 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:16.434046 28581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:53:16.440059 28581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:53:16.449605 28581 catalog_manager.cc:1347] Generated new cluster ID: 130b30c8bfb24fba840e4d3cf060c2a4
I20260812 16:53:16.449769 28581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:53:16.460376 28581 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:53:16.460943 28581 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:53:16.470623 28581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b528c98b345d48baa04127ad1f802d7d: Generated new TSK 0
I20260812 16:53:16.471616 28581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:53:16.487775 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--builtin_ntp_servers=127.8.195.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 16:53:16.862762 28607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:16.863101 28607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:16.863194 28607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:16.870815 28607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:16.870950 28607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:16.871065 28607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:53:16.880604 28607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:16.881964 28607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:16.883796 28607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:16.898221 28617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:16.898424 28619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:16.898252 28623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:16.899121 28607 server_base.cc:1034] running on GCE node
I20260812 16:53:16.899808 28607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:16.901006 28607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:16.902246 28607 hybrid_clock.cc:648] HybridClock initialized: now 1786553596902167 us; error 101 us; skew 500 ppm
I20260812 16:53:16.902558 28607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:16.905098 28607 webserver.cc:460] Webserver started at http://127.8.195.129:38339/ using document root <none> and password file <none>
I20260812 16:53:16.905889 28607 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:16.905985 28607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:16.906265 28607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:16.911909 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "2a9b24c93f10493ab07253b9f0a9fef9"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "7784c9cf823995dc400c904a573e05c1"
server_key_iv: "485b57f59f53bdc9c436e5df2852bec3"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.912768 28607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "2a9b24c93f10493ab07253b9f0a9fef9"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "7784c9cf823995dc400c904a573e05c1"
server_key_iv: "485b57f59f53bdc9c436e5df2852bec3"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.918519 28607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 16:53:16.922226 28632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:16.923812 28607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:53:16.923964 28607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "2a9b24c93f10493ab07253b9f0a9fef9"
format_stamp: "Formatted at 2026-08-12 16:53:16 on dist-test-slave-z8x9"
server_key: "7784c9cf823995dc400c904a573e05c1"
server_key_iv: "485b57f59f53bdc9c436e5df2852bec3"
server_key_version: "encryptionkey@0"
I20260812 16:53:16.924160 28607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:16.930258 28607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:16.931069 28607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:16.931308 28607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:16.932250 28607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:16.933946 28607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:16.934043 28607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:16.934195 28607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:16.934299 28607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:16.971334 28607 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:42929
I20260812 16:53:16.971439 28848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:42929 every 8 connection(s)
I20260812 16:53:16.972834 28607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:53:16.981150 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 28607
I20260812 16:53:16.981451 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:53:16.982147 8974 external_mini_cluster.cc:1428] Setting key 5daee3e5a813bff66a26ba607d142feb
I20260812 16:53:16.982828 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:16.976227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:46853 (local address 127.8.195.190:41023)
0812 16:53:16.976757 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:16.976762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:16.978172 (+ 1410us) server_negotiation.cc:407] Connection header received
0812 16:53:16.980034 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:16.980041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:16.980127 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:16.980258 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:16.981930 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:16.981942 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:16.981945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:16.981965 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:16.982012 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:16.982024 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:16.982172 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:16.982398 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 16:53:16.982478 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":285,"threads_started":1}
I20260812 16:53:16.983875 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:16.976914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41023 (local address 127.8.195.129:46853)
0812 16:53:16.977984 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:16.978025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:16.979713 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:16.980539 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:16.980548 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:16.980654 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:16.981574 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:16.981578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:16.981632 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:16.981732 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:16.982125 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:16.982137 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:16.982147 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:53:16.982489 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 16:53:16.982784 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":349,"threads_started":1}
I20260812 16:53:16.985291 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--builtin_ntp_servers=127.8.195.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 16:53:16.985700 28850 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41023
I20260812 16:53:16.986042 28850 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:16.986689 28850 heartbeater.cc:511] Master 127.8.195.190:41023 requested a full tablet report, sending...
I20260812 16:53:16.989319 28485 ts_manager.cc:194] Registered new tserver with Master: 2a9b24c93f10493ab07253b9f0a9fef9 (127.8.195.129:42929)
W20260812 16:53:17.341341 28865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:17.341673 28865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:17.341768 28865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:17.349466 28865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:17.349592 28865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:17.349705 28865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:53:17.360146 28865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:17.361732 28865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:17.363646 28865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:17.378554 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:17.378645 28873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:17.378975 28871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:17.379354 28865 server_base.cc:1034] running on GCE node
I20260812 16:53:17.380052 28865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:17.381129 28865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:17.382403 28865 hybrid_clock.cc:648] HybridClock initialized: now 1786553597382352 us; error 51 us; skew 500 ppm
I20260812 16:53:17.382781 28865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:17.385470 28865 webserver.cc:460] Webserver started at http://127.8.195.130:40589/ using document root <none> and password file <none>
I20260812 16:53:17.386229 28865 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:17.386380 28865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:17.386793 28865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:17.389889 28865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "205f37f2642f4454ae06bad31dbd0f85"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "2ed83b7b59df32cd31dc8ead7bfe950e"
server_key_iv: "ebea9a625320d50694d9ead1fdab0496"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.390774 28865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "205f37f2642f4454ae06bad31dbd0f85"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "2ed83b7b59df32cd31dc8ead7bfe950e"
server_key_iv: "ebea9a625320d50694d9ead1fdab0496"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.397162 28865 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 16:53:17.401206 28891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.402874 28865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:53:17.403095 28865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "205f37f2642f4454ae06bad31dbd0f85"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "2ed83b7b59df32cd31dc8ead7bfe950e"
server_key_iv: "ebea9a625320d50694d9ead1fdab0496"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.403385 28865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:17.409929 28865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:17.410817 28865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:17.411111 28865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:17.412104 28865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:17.413678 28865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:17.413810 28865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.413942 28865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:17.414033 28865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.450610 28865 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:37177
I20260812 16:53:17.450680 29093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:37177 every 8 connection(s)
I20260812 16:53:17.452039 28865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:53:17.458331 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 28865
I20260812 16:53:17.458573 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:53:17.459187 8974 external_mini_cluster.cc:1428] Setting key 04f2115173f518e71bf6a48751d4bf24
I20260812 16:53:17.461512 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:17.454890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:45231 (local address 127.8.195.190:41023)
0812 16:53:17.455114 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:17.455120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:17.456660 (+ 1540us) server_negotiation.cc:407] Connection header received
0812 16:53:17.458541 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:17.458547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:17.458631 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:17.458755 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:17.460668 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:17.460674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:17.460698 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:17.460718 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:17.460752 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:17.460764 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:17.460926 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:17.461129 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 16:53:17.461229 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 16:53:17.462447 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--builtin_ntp_servers=127.8.195.148:44703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 16:53:17.462641 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:17.455475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41023 (local address 127.8.195.130:45231)
0812 16:53:17.456464 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:17.456510 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:17.458229 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:17.459102 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:17.459116 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:17.459271 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:17.460280 (+ 1009us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:17.460283 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:17.460326 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:17.460458 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:17.460876 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:17.460889 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:17.460900 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:53:17.461258 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 16:53:17.461577 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":313,"threads_started":1}
I20260812 16:53:17.464303 29095 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41023
I20260812 16:53:17.464668 29095 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:17.465587 29095 heartbeater.cc:511] Master 127.8.195.190:41023 requested a full tablet report, sending...
I20260812 16:53:17.467342 28485 ts_manager.cc:194] Registered new tserver with Master: 205f37f2642f4454ae06bad31dbd0f85 (127.8.195.130:37177)
W20260812 16:53:17.824831 29099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:17.825263 29099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:17.825433 29099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:17.833490 29099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:17.833665 29099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:17.833865 29099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:53:17.844385 29099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:17.845960 29099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:17.847891 29099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:17.863217 29112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:17.863298 29109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:17.863262 29110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:17.864698 29099 server_base.cc:1034] running on GCE node
I20260812 16:53:17.865582 29099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:17.866731 29099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:17.868001 29099 hybrid_clock.cc:648] HybridClock initialized: now 1786553597867955 us; error 40 us; skew 500 ppm
I20260812 16:53:17.868613 29099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:53:17.871452 29099 webserver.cc:460] Webserver started at http://127.8.195.131:36839/ using document root <none> and password file <none>
I20260812 16:53:17.872290 29099 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:17.872459 29099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:17.872848 29099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:17.876194 29099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "90dd7ad8c521409ab0113b7464c3a548"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "c9483ff4ec1f51c6db7d4a6a83ae7ded"
server_key_iv: "4ab09ed37690950bde1d7e701a3a4517"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.877118 29099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "90dd7ad8c521409ab0113b7464c3a548"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "c9483ff4ec1f51c6db7d4a6a83ae7ded"
server_key_iv: "4ab09ed37690950bde1d7e701a3a4517"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.883311 29099 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 16:53:17.887530 29121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.889676 29099 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:53:17.889911 29099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "90dd7ad8c521409ab0113b7464c3a548"
format_stamp: "Formatted at 2026-08-12 16:53:17 on dist-test-slave-z8x9"
server_key: "c9483ff4ec1f51c6db7d4a6a83ae7ded"
server_key_iv: "4ab09ed37690950bde1d7e701a3a4517"
server_key_version: "encryptionkey@0"
I20260812 16:53:17.890175 29099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:17.896701 29099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:17.897869 29099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:17.898204 29099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:17.899307 29099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:17.900941 29099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:17.901080 29099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.901244 29099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:17.901341 29099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:17.939589 29099 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:44035
I20260812 16:53:17.939654 29319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:44035 every 8 connection(s)
I20260812 16:53:17.940986 29099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:53:17.946852 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 29099
I20260812 16:53:17.947080 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:53:17.947571 8974 external_mini_cluster.cc:1428] Setting key e36215dec6357becf1576040a98457c7
I20260812 16:53:17.950336 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:17.943966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:40489 (local address 127.8.195.190:41023)
0812 16:53:17.944249 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:17.944257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:17.945746 (+ 1489us) server_negotiation.cc:407] Connection header received
0812 16:53:17.947613 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:17.947618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:17.947701 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:17.947838 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:17.949544 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:17.949549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:17.949555 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:17.949580 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:17.949615 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:17.949649 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:17.949776 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:17.950021 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 16:53:17.950077 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 16:53:17.951655 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:17.944551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41023 (local address 127.8.195.131:40489)
0812 16:53:17.945592 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:17.945632 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:17.947307 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:17.948118 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:17.948132 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:17.948280 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:17.949133 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:53:17.949135 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:17.949223 (+ 88us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:53:17.949342 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:17.949765 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:17.949775 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:17.949782 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 16:53:17.950107 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 16:53:17.950467 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":286,"threads_started":1}
I20260812 16:53:17.953388 29321 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41023
I20260812 16:53:17.953755 29321 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:17.954491 29321 heartbeater.cc:511] Master 127.8.195.190:41023 requested a full tablet report, sending...
I20260812 16:53:17.956046 28485 ts_manager.cc:194] Registered new tserver with Master: 90dd7ad8c521409ab0113b7464c3a548 (127.8.195.131:44035)
I20260812 16:53:17.964192 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 16:53:17.971232 28857 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:53:17.969358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.8.195.190:41023)
0812 16:53:17.969549 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:17.969554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:17.970149 (+ 595us) server_negotiation.cc:407] Connection header received
0812 16:53:17.970358 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:17.970363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:17.970426 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:17.970540 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:17.970919 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35298: BlockingRecv error: recv got EOF from 127.0.0.1:35298 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:53:17.971709 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 28607
I20260812 16:53:17.976478 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:17.973160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.8.195.129:42929)
0812 16:53:17.973968 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:17.973983 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:17.974013 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:53:17.974078 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:17.974084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:17.974291 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:17.974484 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:17.975687 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:17.975694 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:17.975706 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:17.975724 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:17.975771 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:17.975785 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:17.975989 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:17.976063 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:53:17.976177 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":504,"threads_started":1}
I20260812 16:53:17.977823 28843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:18.061408 28850 heartbeater.cc:503] Master 127.8.195.190:41023 was elected leader, sending a full tablet report...
I20260812 16:53:18.062837 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 28607
W20260812 16:53:18.087234 28464 connection.cc:535] server connection from 127.8.195.129:46853 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:53:18.087646 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 28865
I20260812 16:53:18.092000 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:18.088847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.8.195.130:37177)
0812 16:53:18.089480 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:18.089488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:18.089518 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:53:18.089587 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:18.089593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:18.089831 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:18.090024 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:18.091183 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:18.091190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:18.091204 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:18.091225 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:18.091279 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:18.091295 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:18.091457 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:18.091595 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 16:53:18.091710 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":255,"threads_started":1}
I20260812 16:53:18.094359 29087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:18.190843 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 28865
I20260812 16:53:18.219138 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 29099
I20260812 16:53:18.223685 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:18.220240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.8.195.131:44035)
0812 16:53:18.220932 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:18.220939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:18.220971 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:53:18.221036 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:18.221043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:18.221367 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:18.221549 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:18.222827 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:18.222834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:18.222846 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:53:18.222864 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:18.222903 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:53:18.222918 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:18.223133 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:18.223227 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 16:53:18.223344 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":367,"threads_started":1}
I20260812 16:53:18.225531 29313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:18.311965 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 29099
I20260812 16:53:18.341487 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 28442
I20260812 16:53:18.342883 28555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:18.451984 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 28442
2026-08-12T16:53:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2629 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:53:18 dist-test-slave-z8x9 krb5kdc[29349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:53:18 dist-test-slave-z8x9 krb5kdc[29349](info): set up 2 sockets
Aug 12 16:53:18 dist-test-slave-z8x9 krb5kdc[29349](info): commencing operation
krb5kdc: starting...
W20260812 16:53:19.889434 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:53:20 dist-test-slave-z8x9 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:53:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:53:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:53:20.218309 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41569
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:41569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:53:20.572206 29372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:20.572602 29372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:20.572790 29372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:20.580273 29372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:53:20.580473 29372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:20.580595 29372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:20.580713 29372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:53:20.580806 29372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:53:20.591173 29372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:41569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:20.592727 29372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:20.594204 29372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:20.606487 29384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:20.606544 29391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:20.606516 29386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:20.607278 29372 server_base.cc:1034] running on GCE node
I20260812 16:53:20.608173 29372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:20.609752 29372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:20.611027 29372 hybrid_clock.cc:648] HybridClock initialized: now 1786553600610982 us; error 48 us; skew 500 ppm
I20260812 16:53:20.611438 29372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:53:20 dist-test-slave-z8x9 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:53:20.618287 29372 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:53:20.620038 29372 webserver.cc:460] Webserver started at http://127.8.195.190:34261/ using document root <none> and password file <none>
I20260812 16:53:20.620820 29372 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:20.620934 29372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:20.621364 29372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:20.624577 29372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "614f42bc0db74a81b00acf04eb7db933"
format_stamp: "Formatted at 2026-08-12 16:53:20 on dist-test-slave-z8x9"
server_key: "7258b035f38dfd04cbd83680c6dcb064"
server_key_iv: "1470e40399a8971cb4092ea08e844ae1"
server_key_version: "encryptionkey@0"
I20260812 16:53:20.625560 29372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "614f42bc0db74a81b00acf04eb7db933"
format_stamp: "Formatted at 2026-08-12 16:53:20 on dist-test-slave-z8x9"
server_key: "7258b035f38dfd04cbd83680c6dcb064"
server_key_iv: "1470e40399a8971cb4092ea08e844ae1"
server_key_version: "encryptionkey@0"
I20260812 16:53:20.631259 29372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:53:20.635265 29407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:20.636768 29372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:53:20.636926 29372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "614f42bc0db74a81b00acf04eb7db933"
format_stamp: "Formatted at 2026-08-12 16:53:20 on dist-test-slave-z8x9"
server_key: "7258b035f38dfd04cbd83680c6dcb064"
server_key_iv: "1470e40399a8971cb4092ea08e844ae1"
server_key_version: "encryptionkey@0"
I20260812 16:53:20.637100 29372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:20.655119 29372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:20.659791 29372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:20.660171 29372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:20.683204 29372 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:41569
I20260812 16:53:20.683236 29494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:41569 every 8 connection(s)
I20260812 16:53:20.684828 29372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:53:20.689541 29495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:20.690299 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 29372
I20260812 16:53:20.690579 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:53:20.691288 8974 external_mini_cluster.cc:1428] Setting key 58729a1fd9a7d72ee1f21caaecf69a4e
I20260812 16:53:20.707185 29495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: Bootstrap starting.
Aug 12 16:53:20 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:53:20.711051 29495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:20.712687 29495 log.cc:824] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:20.716391 29495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: No bootstrap required, opened a new log
I20260812 16:53:20.719802 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:20.693910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38896 (local address 127.8.195.190:41569)
0812 16:53:20.694923 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:20.694948 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:20.695007 (+ 59us) server_negotiation.cc:407] Connection header received
0812 16:53:20.697815 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:20.697858 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:20.698552 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:20.699166 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:20.700757 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:20.703176 (+ 2419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:20.704689 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:20.706008 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:20.706236 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:20.710791 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:20.710808 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:20.710835 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:20.710903 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:20.714971 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:20.715854 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:20.715861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:20.715868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:20.716060 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:20.716503 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:20.716509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:20.716512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:20.717316 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:20.717717 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:20.718187 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 16:53:20.718693 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":457,"threads_started":1}
I20260812 16:53:20.725795 29495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } }
I20260812 16:53:20.726290 29495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:20.726455 29495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 614f42bc0db74a81b00acf04eb7db933, State: Initialized, Role: FOLLOWER
I20260812 16:53:20.727206 29495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } }
I20260812 16:53:20.727520 29495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:53:20.727732 29495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:53:20.727958 29495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:20.730587 29495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } }
I20260812 16:53:20.731144 29495 leader_election.cc:302] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 614f42bc0db74a81b00acf04eb7db933; no voters:
I20260812 16:53:20.731820 29495 leader_election.cc:288] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:53:20.731964 29504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:53:20.733467 29504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [term 1 LEADER]: Becoming Leader. State: Replica: 614f42bc0db74a81b00acf04eb7db933, State: Running, Role: LEADER
I20260812 16:53:20.734114 29504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } }
I20260812 16:53:20.734499 29495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:53:20.737056 29506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 614f42bc0db74a81b00acf04eb7db933. Latest consensus state: current_term: 1 leader_uuid: "614f42bc0db74a81b00acf04eb7db933" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } } }
I20260812 16:53:20.737562 29506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:20.737388 29505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "614f42bc0db74a81b00acf04eb7db933" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614f42bc0db74a81b00acf04eb7db933" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41569 } } }
I20260812 16:53:20.737812 29505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933 [sys.catalog]: This master's current role is: LEADER
I20260812 16:53:20.739691 29516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:53:20.745402 29516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:53:20.754643 29516 catalog_manager.cc:1347] Generated new cluster ID: 78afb69f5a904297a328db51a35c5826
I20260812 16:53:20.754789 29516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:53:20.774375 29516 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:53:20.775799 29516 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:53:20.782006 29516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: Generated new TSK 0
I20260812 16:53:20.782907 29516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:53:20.870543 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:53:21.235785 29538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:21.236143 29538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:21.236251 29538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:21.244077 29538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:21.244199 29538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:21.244321 29538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:53:21.254623 29538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:21.255968 29538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:21.257427 29538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:21.271592 29548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:21.271633 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:21.271658 29557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:21.274772 29538 server_base.cc:1034] running on GCE node
I20260812 16:53:21.275476 29538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:21.276552 29538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:21.277843 29538 hybrid_clock.cc:648] HybridClock initialized: now 1786553601277820 us; error 66 us; skew 500 ppm
I20260812 16:53:21.278224 29538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:53:21 dist-test-slave-z8x9 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:53:21.281971 29538 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:53:21.283789 29538 webserver.cc:460] Webserver started at http://127.8.195.129:44639/ using document root <none> and password file <none>
I20260812 16:53:21.284606 29538 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:21.284735 29538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:21.285120 29538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:21.288350 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "ca786b13f30e47c797a29f1db942e01a"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "ad1dd962a79beb87cc6b7b245930c8f3"
server_key_iv: "f6210e4bc8e767e01376bbdcbebcf68e"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.289273 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "ca786b13f30e47c797a29f1db942e01a"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "ad1dd962a79beb87cc6b7b245930c8f3"
server_key_iv: "f6210e4bc8e767e01376bbdcbebcf68e"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.294893 29538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 16:53:21.298887 29564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.300383 29538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:53:21.300587 29538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "ca786b13f30e47c797a29f1db942e01a"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "ad1dd962a79beb87cc6b7b245930c8f3"
server_key_iv: "f6210e4bc8e767e01376bbdcbebcf68e"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.300813 29538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:21.345875 29538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:21.350193 29538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:21.350567 29538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:21.351619 29538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:21.353142 29538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:21.353236 29538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.353369 29538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:21.353436 29538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.398366 29538 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:40241
I20260812 16:53:21.398422 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:40241 every 8 connection(s)
I20260812 16:53:21.399744 29538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:53:21.405292 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 29538
I20260812 16:53:21.405575 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:53:21.406418 8974 external_mini_cluster.cc:1428] Setting key 8737f3488db1c1ade641510e731ae2d9
Aug 12 16:53:21 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:53:21.425239 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:21.402648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:38717 (local address 127.8.195.190:41569)
0812 16:53:21.403163 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:21.403167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:21.404047 (+ 880us) server_negotiation.cc:407] Connection header received
0812 16:53:21.406964 (+ 2917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:21.406969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:21.407045 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:21.407147 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:21.409268 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.410496 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.412959 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.413705 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.413863 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:21.417862 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:21.417871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:21.417879 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:21.417926 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:21.421570 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:21.422684 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:21.422688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:21.422690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:21.422814 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:21.423392 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:21.423395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:21.423397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:21.423661 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:21.423763 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:21.424743 (+ 980us) server_negotiation.cc:299] Negotiation successful
0812 16:53:21.424921 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":294,"threads_started":1}
I20260812 16:53:21.426949 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:21.403138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41569 (local address 127.8.195.129:38717)
0812 16:53:21.403869 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:21.403909 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:21.406602 (+ 2693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:21.407459 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:21.407468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:21.408243 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:21.409051 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:21.409071 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.410672 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.410680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:21.412799 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:21.412814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.413925 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.413930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:21.414141 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:21.414936 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:21.414966 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:21.417607 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:21.421748 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:21.421761 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:21.421785 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:21.422548 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:21.422947 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:21.422955 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:21.422962 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:21.423278 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:21.423812 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:21.423824 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:21.424325 (+ 501us) client_negotiation.cc:769] Sending connection context
0812 16:53:21.424663 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 16:53:21.425328 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":283,"threads_started":1}
I20260812 16:53:21.428653 29779 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41569
I20260812 16:53:21.428987 29779 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:21.429808 29779 heartbeater.cc:511] Master 127.8.195.190:41569 requested a full tablet report, sending...
I20260812 16:53:21.431924 29439 ts_manager.cc:194] Registered new tserver with Master: ca786b13f30e47c797a29f1db942e01a (127.8.195.129:40241)
I20260812 16:53:21.433511 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:38717
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:53:21.482156 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:53:21.835841 29796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:21.836163 29796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:21.836277 29796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:21.844000 29796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:21.844129 29796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:21.844255 29796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:53:21.854547 29796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:21.855914 29796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:21.857316 29796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:21.872280 29812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:21.872345 29807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:21.872309 29806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:21.872980 29796 server_base.cc:1034] running on GCE node
I20260812 16:53:21.873759 29796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:21.874842 29796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:21.876178 29796 hybrid_clock.cc:648] HybridClock initialized: now 1786553601876072 us; error 60 us; skew 500 ppm
I20260812 16:53:21.876590 29796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:53:21 dist-test-slave-z8x9 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:53:21.880380 29796 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:53:21.882087 29796 webserver.cc:460] Webserver started at http://127.8.195.130:45441/ using document root <none> and password file <none>
I20260812 16:53:21.882917 29796 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:21.883075 29796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:21.883406 29796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:21.886600 29796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "16f2b5464b684207891184c8d2ddec08"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "5554cc6c426c06ba7d13c067942acddc"
server_key_iv: "469c2172a9bf5b3eda067868ac3e77e2"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.887451 29796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "16f2b5464b684207891184c8d2ddec08"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "5554cc6c426c06ba7d13c067942acddc"
server_key_iv: "469c2172a9bf5b3eda067868ac3e77e2"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.893108 29796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:53:21.897334 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.898862 29796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:53:21.899091 29796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "16f2b5464b684207891184c8d2ddec08"
format_stamp: "Formatted at 2026-08-12 16:53:21 on dist-test-slave-z8x9"
server_key: "5554cc6c426c06ba7d13c067942acddc"
server_key_iv: "469c2172a9bf5b3eda067868ac3e77e2"
server_key_version: "encryptionkey@0"
I20260812 16:53:21.899328 29796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:21.914880 29796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:21.919422 29796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:21.919744 29796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:21.920746 29796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:21.922209 29796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:21.922327 29796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.922478 29796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:21.922559 29796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:21.957643 29796 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:41451
I20260812 16:53:21.957686 30056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:41451 every 8 connection(s)
I20260812 16:53:21.959013 29796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:53:21.964586 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 29796
I20260812 16:53:21.964859 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:53:21.965448 8974 external_mini_cluster.cc:1428] Setting key 7f7ee64668462c905739ea4dbe00e7f6
Aug 12 16:53:21 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:53:21.984985 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:21.961893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:56213 (local address 127.8.195.190:41569)
0812 16:53:21.962387 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:21.962392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:21.963337 (+ 945us) server_negotiation.cc:407] Connection header received
0812 16:53:21.966230 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:21.966238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:21.966345 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:21.966482 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:21.968830 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.969740 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.972506 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.972971 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.973086 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:21.978268 (+ 5182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:21.978274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:21.978278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:21.978319 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:21.981256 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:21.982160 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:21.982167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:21.982172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:21.982406 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:21.982855 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:21.982863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:21.982868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:21.983289 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:21.983463 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:21.984192 (+ 729us) server_negotiation.cc:299] Negotiation successful
0812 16:53:21.984485 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":260,"threads_started":1}
I20260812 16:53:21.985733 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:21.962410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41569 (local address 127.8.195.130:56213)
0812 16:53:21.963152 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:21.963192 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:21.965786 (+ 2594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:21.966869 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:21.966880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:21.967633 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:21.968557 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:21.968592 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.969942 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.969951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:21.972302 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:21.972324 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:21.973279 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:21.973289 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:21.973655 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:21.974579 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:21.974616 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:21.977934 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:21.981448 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:21.981462 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:21.981485 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:21.982013 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:21.982528 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:21.982531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:21.982533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:21.982737 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:21.983413 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:21.983419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:21.983773 (+ 354us) client_negotiation.cc:769] Sending connection context
0812 16:53:21.984039 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 16:53:21.984474 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":299,"threads_started":1}
I20260812 16:53:21.987349 30058 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41569
I20260812 16:53:21.987718 30058 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:21.988512 30058 heartbeater.cc:511] Master 127.8.195.190:41569 requested a full tablet report, sending...
I20260812 16:53:21.990207 29439 ts_manager.cc:194] Registered new tserver with Master: 16f2b5464b684207891184c8d2ddec08 (127.8.195.130:41451)
I20260812 16:53:21.991101 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:56213
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:53:22.037515 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:53:22.388695 30068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:53:22.389034 30068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:53:22.389262 30068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:53:22.397408 30068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:53:22.397572 30068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:53:22.397807 30068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:53:22.408550 30068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:53:22.410018 30068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:53:22.411404 30068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:53:22.425766 30078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:22.425710 30079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:53:22.426262 30083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:53:22.426716 30068 server_base.cc:1034] running on GCE node
I20260812 16:53:22.427698 30068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:53:22.428844 30068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:53:22.430137 30068 hybrid_clock.cc:648] HybridClock initialized: now 1786553602430080 us; error 46 us; skew 500 ppm
I20260812 16:53:22.430569 30068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:53:22 dist-test-slave-z8x9 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:53:22.434559 30068 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:53:22.436486 30068 webserver.cc:460] Webserver started at http://127.8.195.131:37233/ using document root <none> and password file <none>
I20260812 16:53:22.436769 29779 heartbeater.cc:503] Master 127.8.195.190:41569 was elected leader, sending a full tablet report...
I20260812 16:53:22.437368 30068 fs_manager.cc:359] Metadata directory not provided
I20260812 16:53:22.437531 30068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:53:22.437852 30068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:53:22.441322 30068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "54573353764340d581919c28cb98316d"
format_stamp: "Formatted at 2026-08-12 16:53:22 on dist-test-slave-z8x9"
server_key: "0d2adcfa1ee5d134969248ac133ed684"
server_key_iv: "fee2492a9cf8d2a086309309328340a1"
server_key_version: "encryptionkey@0"
I20260812 16:53:22.442322 30068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "54573353764340d581919c28cb98316d"
format_stamp: "Formatted at 2026-08-12 16:53:22 on dist-test-slave-z8x9"
server_key: "0d2adcfa1ee5d134969248ac133ed684"
server_key_iv: "fee2492a9cf8d2a086309309328340a1"
server_key_version: "encryptionkey@0"
I20260812 16:53:22.448032 30068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:53:22.452055 30091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:22.453658 30068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:53:22.453861 30068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "54573353764340d581919c28cb98316d"
format_stamp: "Formatted at 2026-08-12 16:53:22 on dist-test-slave-z8x9"
server_key: "0d2adcfa1ee5d134969248ac133ed684"
server_key_iv: "fee2492a9cf8d2a086309309328340a1"
server_key_version: "encryptionkey@0"
I20260812 16:53:22.454095 30068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:53:22.501915 30068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:53:22.506963 30068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:53:22.507299 30068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:53:22.508347 30068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:53:22.509858 30068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:53:22.509963 30068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:22.510124 30068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:53:22.510198 30068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:53:22.546731 30068 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:36325
I20260812 16:53:22.546770 30306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:36325 every 8 connection(s)
I20260812 16:53:22.548132 30068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:53:22.550539 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 30068
I20260812 16:53:22.550756 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:53:22.551258 8974 external_mini_cluster.cc:1428] Setting key 2700f6d034cffb1ebcb862863914fcae
Aug 12 16:53:22 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:53:22.570683 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.551056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:54031 (local address 127.8.195.190:41569)
0812 16:53:22.551615 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.551622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.552799 (+ 1177us) server_negotiation.cc:407] Connection header received
0812 16:53:22.555175 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.555179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.555253 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.555352 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:22.557012 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.557964 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.559828 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.560240 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.560342 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.564034 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:22.564040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:22.564043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:22.564079 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:22.567634 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.568441 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.568445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.568447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.568570 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.568980 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.568984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.568987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.569332 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:22.569482 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.570205 (+ 723us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.570406 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
I20260812 16:53:22.571741 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.551745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41569 (local address 127.8.195.131:54031)
0812 16:53:22.552621 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.552677 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.554897 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.555642 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.555651 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.556217 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:53:22.556836 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.556855 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.558166 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.558170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.559658 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.559672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.560466 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.560471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.560683 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.561475 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:53:22.561502 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:53:22.563812 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:53:22.567800 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:22.567808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:22.567822 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:22.568314 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:22.568683 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:53:22.568686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:53:22.568689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:53:22.568884 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:53:22.569474 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:53:22.569480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:53:22.569861 (+ 381us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.570108 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.570539 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":375,"threads_started":1}
I20260812 16:53:22.573331 30307 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41569
I20260812 16:53:22.573678 30307 heartbeater.cc:464] Registering TS with master...
I20260812 16:53:22.574476 30307 heartbeater.cc:511] Master 127.8.195.190:41569 requested a full tablet report, sending...
I20260812 16:53:22.576139 29439 ts_manager.cc:194] Registered new tserver with Master: 54573353764340d581919c28cb98316d (127.8.195.131:36325)
I20260812 16:53:22.577162 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:54031
I20260812 16:53:22.580269 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:53:22.598701 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.585331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.8.195.190:41569)
0812 16:53:22.585522 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.585527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.586150 (+ 623us) server_negotiation.cc:407] Connection header received
0812 16:53:22.586221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.586225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.586312 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.586415 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:22.587935 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.588696 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.590024 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.590474 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.590586 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.593055 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:22.593060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:22.593063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:22.593102 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:22.596046 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.596778 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.596782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.596785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.596928 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.597383 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.597387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.597389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.597757 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:22.597972 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.598272 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.598459 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:53:22.603341 29439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:53:22.610666 29439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:53:22.635286 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.623670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:36325 (local address 127.0.0.1:33784)
0812 16:53:22.624568 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.624621 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.624863 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.626320 (+ 1457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.626327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.626362 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:53:22.626976 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.626989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.629408 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.629414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.632016 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.632033 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.634093 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.634100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.634320 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.634386 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.634538 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.634767 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":25,"thread_start_us":391,"threads_started":1}
I20260812 16:53:22.636018 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.624725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.8.195.130:41451)
0812 16:53:22.626511 (+ 1786us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.626520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.627313 (+ 793us) server_negotiation.cc:407] Connection header received
0812 16:53:22.627437 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.627445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.627712 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.627912 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:53:22.628775 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.630372 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.632947 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.634515 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.634643 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.634839 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.635482 (+ 643us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.635670 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1628,"thread_start_us":1506,"threads_started":1}
I20260812 16:53:22.636827 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.624134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41451 (local address 127.0.0.1:40854)
0812 16:53:22.627045 (+ 2911us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.627065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.627187 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.628150 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.628154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.628180 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:53:22.628584 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.628597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.630543 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.630549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.632760 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.632776 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.634997 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.635003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.635203 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.635248 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.636221 (+ 973us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.636401 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"mutex_wait_us":1,"thread_start_us":2554,"threads_started":1}
I20260812 16:53:22.637434 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.624049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.8.195.131:36325)
0812 16:53:22.625602 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.625612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.625647 (+ 35us) server_negotiation.cc:407] Connection header received
0812 16:53:22.625750 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.625759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.626035 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.627228 (+ 1193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:53:22.627545 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.629253 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.632274 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.633852 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.636443 (+ 2591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.636745 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.636843 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.637030 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1387,"thread_start_us":1234,"threads_started":1}
I20260812 16:53:22.641636 30179 tablet_service.cc:1467] Processing CreateTablet for tablet 510291c878c8429e8638392712af83b7 (DEFAULT_TABLE table=test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:53:22.641629 29940 tablet_service.cc:1467] Processing CreateTablet for tablet 510291c878c8429e8638392712af83b7 (DEFAULT_TABLE table=test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:53:22.643143 30179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510291c878c8429e8638392712af83b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:22.643177 29940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510291c878c8429e8638392712af83b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:22.648743 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.626461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:40241 (local address 127.0.0.1:60766)
0812 16:53:22.635556 (+ 9095us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.635577 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.635677 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.640286 (+ 4609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.640291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.640323 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:53:22.640498 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.640509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.643612 (+ 3103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.643618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.645542 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.645560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.647848 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.647855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.648046 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.648088 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.648181 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.648402 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8992,"thread_start_us":13263,"threads_started":1}
I20260812 16:53:22.649679 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.627264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.8.195.129:40241)
0812 16:53:22.639169 (+ 11905us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.639182 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.639214 (+ 32us) server_negotiation.cc:407] Connection header received
0812 16:53:22.639321 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.639328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.640053 (+ 725us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.640269 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:53:22.640768 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.643415 (+ 2647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.645781 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.647348 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.647512 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.648994 (+ 1482us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.649095 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.649310 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11687,"thread_start_us":13245,"threads_started":1}
I20260812 16:53:22.654009 29658 tablet_service.cc:1467] Processing CreateTablet for tablet 510291c878c8429e8638392712af83b7 (DEFAULT_TABLE table=test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:53:22.655840 29658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 510291c878c8429e8638392712af83b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:53:22.657368 30335 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Bootstrap starting.
I20260812 16:53:22.658219 30340 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Bootstrap starting.
I20260812 16:53:22.662364 30340 tablet_bootstrap.cc:654] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:22.662398 30335 tablet_bootstrap.cc:654] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:22.663851 30340 log.cc:824] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:22.663865 30335 log.cc:824] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:22.674912 30335 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: No bootstrap required, opened a new log
I20260812 16:53:22.675282 30335 ts_tablet_manager.cc:1397] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260812 16:53:22.677623 30351 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Bootstrap starting.
I20260812 16:53:22.678715 30340 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: No bootstrap required, opened a new log
I20260812 16:53:22.679032 30340 ts_tablet_manager.cc:1397] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.010s
I20260812 16:53:22.681668 30351 tablet_bootstrap.cc:654] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Neither blocks nor log segments found. Creating new log.
I20260812 16:53:22.683943 30351 log.cc:824] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:53:22.684816 30335 raft_consensus.cc:348] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.685683 30335 raft_consensus.cc:374] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:22.685874 30335 raft_consensus.cc:729] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f2b5464b684207891184c8d2ddec08, State: Initialized, Role: FOLLOWER
I20260812 16:53:22.686967 30335 consensus_queue.cc:260] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.690474 30340 raft_consensus.cc:348] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.691098 30340 raft_consensus.cc:374] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:22.691233 30340 raft_consensus.cc:729] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54573353764340d581919c28cb98316d, State: Initialized, Role: FOLLOWER
I20260812 16:53:22.692018 30340 consensus_queue.cc:260] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.693820 30307 heartbeater.cc:503] Master 127.8.195.190:41569 was elected leader, sending a full tablet report...
I20260812 16:53:22.695658 30351 tablet_bootstrap.cc:492] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: No bootstrap required, opened a new log
I20260812 16:53:22.696031 30351 ts_tablet_manager.cc:1397] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.006s
I20260812 16:53:22.696184 30340 ts_tablet_manager.cc:1428] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Time spent starting tablet: real 0.017s user 0.008s sys 0.007s
I20260812 16:53:22.697275 30058 heartbeater.cc:503] Master 127.8.195.190:41569 was elected leader, sending a full tablet report...
I20260812 16:53:22.699719 30335 ts_tablet_manager.cc:1428] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Time spent starting tablet: real 0.024s user 0.013s sys 0.007s
I20260812 16:53:22.707612 30351 raft_consensus.cc:348] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.708041 30351 raft_consensus.cc:374] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:53:22.708184 30351 raft_consensus.cc:729] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca786b13f30e47c797a29f1db942e01a, State: Initialized, Role: FOLLOWER
I20260812 16:53:22.708772 30351 consensus_queue.cc:260] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.710583 30351 ts_tablet_manager.cc:1428] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
W20260812 16:53:22.711190 30059 tablet.cc:2369] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:53:22.733291 30362 raft_consensus.cc:482] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:53:22.733631 30362 raft_consensus.cc:504] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.735423 30362 leader_election.cc:288] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54573353764340d581919c28cb98316d (127.8.195.131:36325), 16f2b5464b684207891184c8d2ddec08 (127.8.195.130:41451)
I20260812 16:53:22.742929 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.735994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:45825 (local address 127.8.195.130:41451)
0812 16:53:22.736238 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.736247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.737067 (+ 820us) server_negotiation.cc:407] Connection header received
0812 16:53:22.737374 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.737378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.737440 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.737542 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:53:22.738146 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.738993 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.741019 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.741928 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.742133 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.742214 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.742496 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.742631 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 16:53:22.743050 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.735989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41451 (local address 127.8.195.129:45825)
0812 16:53:22.736909 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.736926 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.737153 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.737559 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.737563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.737588 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:53:22.737987 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.738001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.739118 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.739126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.740850 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.740864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.742148 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.742154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.742323 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.742354 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.742469 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.742620 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":28,"thread_start_us":562,"threads_started":1}
I20260812 16:53:22.743311 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.736133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49759 (local address 127.8.195.131:36325)
0812 16:53:22.736363 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.736371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.737257 (+ 886us) server_negotiation.cc:407] Connection header received
0812 16:53:22.737513 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.737517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.737579 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.737683 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:53:22.738350 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.739169 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.741320 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.742325 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.742513 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.742591 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.742903 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.743034 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:53:22.743495 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.736034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:36325 (local address 127.8.195.129:49759)
0812 16:53:22.737090 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0812 16:53:22.737117 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:53:22.737357 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:53:22.737702 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:53:22.737706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:53:22.737831 (+ 125us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:53:22.738169 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.738178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.739341 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.739346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.741126 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:53:22.741140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.742549 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.742556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:53:22.742727 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.742760 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 16:53:22.742872 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:53:22.743014 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":46,"spinlock_wait_cycles":27520,"thread_start_us":831,"threads_started":1}
I20260812 16:53:22.744546 29968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510291c878c8429e8638392712af83b7" candidate_uuid: "ca786b13f30e47c797a29f1db942e01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f2b5464b684207891184c8d2ddec08" is_pre_election: true
I20260812 16:53:22.745129 29968 raft_consensus.cc:2454] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca786b13f30e47c797a29f1db942e01a in term 0.
I20260812 16:53:22.745502 30215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510291c878c8429e8638392712af83b7" candidate_uuid: "ca786b13f30e47c797a29f1db942e01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54573353764340d581919c28cb98316d" is_pre_election: true
I20260812 16:53:22.746013 29568 leader_election.cc:302] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f2b5464b684207891184c8d2ddec08, ca786b13f30e47c797a29f1db942e01a; no voters:
I20260812 16:53:22.746037 30215 raft_consensus.cc:2454] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca786b13f30e47c797a29f1db942e01a in term 0.
I20260812 16:53:22.746444 30362 raft_consensus.cc:2781] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:53:22.746630 30362 raft_consensus.cc:482] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:53:22.746757 30362 raft_consensus.cc:3037] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:22.749086 30362 raft_consensus.cc:504] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.749830 30362 leader_election.cc:288] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [CANDIDATE]: Term 1 election: Requested vote from peers 54573353764340d581919c28cb98316d (127.8.195.131:36325), 16f2b5464b684207891184c8d2ddec08 (127.8.195.130:41451)
I20260812 16:53:22.750412 30215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510291c878c8429e8638392712af83b7" candidate_uuid: "ca786b13f30e47c797a29f1db942e01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54573353764340d581919c28cb98316d"
I20260812 16:53:22.750458 29968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "510291c878c8429e8638392712af83b7" candidate_uuid: "ca786b13f30e47c797a29f1db942e01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16f2b5464b684207891184c8d2ddec08"
I20260812 16:53:22.750665 30215 raft_consensus.cc:3037] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:22.750669 29968 raft_consensus.cc:3037] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:53:22.753115 30215 raft_consensus.cc:2454] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca786b13f30e47c797a29f1db942e01a in term 1.
I20260812 16:53:22.753254 29968 raft_consensus.cc:2454] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca786b13f30e47c797a29f1db942e01a in term 1.
I20260812 16:53:22.753780 29566 leader_election.cc:302] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54573353764340d581919c28cb98316d, ca786b13f30e47c797a29f1db942e01a; no voters:
I20260812 16:53:22.754156 30362 raft_consensus.cc:2781] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:53:22.754678 30362 raft_consensus.cc:686] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 1 LEADER]: Becoming Leader. State: Replica: ca786b13f30e47c797a29f1db942e01a, State: Running, Role: LEADER
I20260812 16:53:22.755312 30362 consensus_queue.cc:237] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } }
I20260812 16:53:22.760612 29439 catalog_manager.cc:5495] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a reported cstate change: term changed from 0 to 1, leader changed from <none> to ca786b13f30e47c797a29f1db942e01a (127.8.195.129). New cstate: current_term: 1 leader_uuid: "ca786b13f30e47c797a29f1db942e01a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca786b13f30e47c797a29f1db942e01a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 40241 } health_report { overall_health: HEALTHY } } }
I20260812 16:53:22.799877 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.794367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.8.195.129:40241)
0812 16:53:22.794659 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.794664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.794685 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:53:22.794881 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.794885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.794969 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.795097 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:53:22.795560 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.796293 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.797792 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.798202 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.798309 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.798818 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:53:22.799359 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:53:22.799480 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.799599 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.799669 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260812 16:53:22.800138 30309 tablet.cc:2369] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:53:22.808753 29968 raft_consensus.cc:1261] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 1 FOLLOWER]: Refusing update from remote peer ca786b13f30e47c797a29f1db942e01a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:53:22.808748 30215 raft_consensus.cc:1261] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 1 FOLLOWER]: Refusing update from remote peer ca786b13f30e47c797a29f1db942e01a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:53:22.809841 30365 consensus_queue.cc:1035] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "54573353764340d581919c28cb98316d" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:53:22.810258 30362 consensus_queue.cc:1035] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f2b5464b684207891184c8d2ddec08" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:53:22.838932 29438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:53:22.839383 29438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:53:22.843863 29438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 614f42bc0db74a81b00acf04eb7db933: Sending DeleteTablet for 3 replicas of tablet 510291c878c8429e8638392712af83b7
I20260812 16:53:22.845342 29940 tablet_service.cc:1514] Processing DeleteTablet for tablet 510291c878c8429e8638392712af83b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:53:22 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:40854
I20260812 16:53:22.845374 30179 tablet_service.cc:1514] Processing DeleteTablet for tablet 510291c878c8429e8638392712af83b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:53:22 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:33784
I20260812 16:53:22.846417 30391 tablet_replica.cc:330] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: stopping tablet replica
I20260812 16:53:22.846691 29658 tablet_service.cc:1514] Processing DeleteTablet for tablet 510291c878c8429e8638392712af83b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:53:22 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:60766
I20260812 16:53:22.846974 30391 raft_consensus.cc:2229] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:53:22.847558 30391 raft_consensus.cc:2258] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:53:22.848469 30392 tablet_replica.cc:330] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: stopping tablet replica
I20260812 16:53:22.849068 30392 raft_consensus.cc:2229] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:53:22.849298 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 29538
I20260812 16:53:22.849783 30392 raft_consensus.cc:2258] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:53:22.852607 30392 ts_tablet_manager.cc:1905] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:53:22.858069 30395 tablet_replica.cc:330] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: stopping tablet replica
I20260812 16:53:22.858759 30395 raft_consensus.cc:2229] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:53:22.859509 30395 raft_consensus.cc:2258] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:53:22.858111 30391 ts_tablet_manager.cc:1905] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:53:22.861723 30392 ts_tablet_manager.cc:1918] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:53:22.861954 30392 log.cc:1196] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/510291c878c8429e8638392712af83b7
I20260812 16:53:22.861948 30395 ts_tablet_manager.cc:1905] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:53:22.862607 30392 ts_tablet_manager.cc:1939] T 510291c878c8429e8638392712af83b7 P 54573353764340d581919c28cb98316d: Deleting consensus metadata
I20260812 16:53:22.864600 29410 catalog_manager.cc:4841] TS 54573353764340d581919c28cb98316d (127.8.195.131:36325): tablet 510291c878c8429e8638392712af83b7 (table test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]) successfully deleted
I20260812 16:53:22.870628 30395 ts_tablet_manager.cc:1918] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:53:22.870985 30395 log.cc:1196] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/510291c878c8429e8638392712af83b7
I20260812 16:53:22.870957 30391 ts_tablet_manager.cc:1918] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:53:22.871152 30391 log.cc:1196] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/510291c878c8429e8638392712af83b7
I20260812 16:53:22.871729 30395 ts_tablet_manager.cc:1939] T 510291c878c8429e8638392712af83b7 P ca786b13f30e47c797a29f1db942e01a: Deleting consensus metadata
I20260812 16:53:22.871830 30391 ts_tablet_manager.cc:1939] T 510291c878c8429e8638392712af83b7 P 16f2b5464b684207891184c8d2ddec08: Deleting consensus metadata
Aug 12 16:53:22 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:53:22.874007 29412 catalog_manager.cc:4841] TS 16f2b5464b684207891184c8d2ddec08 (127.8.195.130:41451): tablet 510291c878c8429e8638392712af83b7 (table test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]) successfully deleted
I20260812 16:53:22.874686 29408 catalog_manager.cc:4841] TS ca786b13f30e47c797a29f1db942e01a (127.8.195.129:40241): tablet 510291c878c8429e8638392712af83b7 (table test-table [id=c7e516b397fa4f8ba61fa3e8dff96498]) successfully deleted
I20260812 16:53:22.881493 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:22.850797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.8.195.129:40241)
0812 16:53:22.850982 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:22.850989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:22.861761 (+ 10772us) server_negotiation.cc:407] Connection header received
0812 16:53:22.862018 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:22.862023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:22.862109 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:22.862234 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:22.863951 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.865098 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.867035 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:22.867835 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:22.867986 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:22.874773 (+ 6787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:22.874786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:22.874829 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:22.874902 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:22.878306 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.879115 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.879124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.879126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.879294 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:22.879743 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:22.879748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:22.879753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:22.880150 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:22.880484 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:22.880813 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 16:53:22.881087 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:53:22.882557 29770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:22.988238 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 29538
I20260812 16:53:23.018836 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 29796
Aug 12 16:53:23 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:53:23.035235 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:23.019874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.8.195.130:41451)
0812 16:53:23.020093 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:23.020099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:23.020123 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:53:23.020299 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:23.020303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:23.020371 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:23.020483 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:23.021903 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:23.022696 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:23.023967 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:23.024377 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:23.024554 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:23.028990 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:23.029007 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:23.029024 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:23.029068 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:23.032058 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:23.032819 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:23.032828 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:23.032830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:23.032990 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:23.033558 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:23.033562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:23.033564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:23.034018 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:23.034275 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:23.034720 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 16:53:23.034931 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 16:53:23.036099 30049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:23.136694 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 29796
I20260812 16:53:23.162580 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 30068
Aug 12 16:53:23 dist-test-slave-z8x9 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:53:23.179428 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:53:23.163395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.8.195.131:36325)
0812 16:53:23.163594 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:53:23.163603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:53:23.163647 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:53:23.163896 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:53:23.163903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:53:23.163984 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:53:23.164096 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:53:23.165574 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:23.166338 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:23.167977 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:53:23.168402 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:53:23.168544 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:53:23.173364 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:53:23.173385 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:53:23.173400 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:53:23.173445 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:53:23.176461 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:23.177317 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:23.177330 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:23.177337 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:23.177511 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:53:23.177874 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:53:23.177878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:53:23.177880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:53:23.178296 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:53:23.178518 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 16:53:23.178828 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 16:53:23.179009 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:53:23.180198 30300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:23.273226 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 30068
I20260812 16:53:23.300628 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 29372
I20260812 16:53:23.301977 29488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:53:23.414637 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 29372
2026-08-12T16:53:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4965 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:53:23 dist-test-slave-z8x9 krb5kdc[30406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:53:23 dist-test-slave-z8x9 krb5kdc[30406](info): set up 2 sockets
Aug 12 16:53:23 dist-test-slave-z8x9 krb5kdc[30406](info): commencing operation
krb5kdc: starting...
W20260812 16:53:24.874387 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.404s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:53:25 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:53:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:53:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangeradmin_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangeradmin_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangerlookup_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangerlookup_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
I20260812 16:53:25.338061 8974 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKFRmA3/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/postgres -l logfile start
2026-08-12 16:53:28.131 UTC [30457] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:53:28.131 UTC [30457] LOG: listening on IPv4 address "127.8.195.148", port 51201
2026-08-12 16:53:28.135 UTC [30457] LOG: listening on Unix socket "/tmp/.s.PGSQL.51201"
2026-08-12 16:53:28.161 UTC [30463] LOG: database system was shut down at 2026-08-12 16:53:26 UTC
2026-08-12 16:53:28.165 UTC [30457] LOG: database system is ready to accept connections
I20260812 16:53:29.439169 8974 mini_postgres.cc:96] Postgres bound to 51201
2026-08-12 16:53:29.466 UTC [30484] FATAL: database "slave" does not exist
127.8.195.148:51201 - accepting connections
I20260812 16:53:29.467595 8974 mini_ranger.cc:161] Starting Ranger...
I20260812 16:53:29.504177 8974 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:53:30.045717 8974 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:53:30.045883 8974 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin
2026-08-12 16:53:30,391 [I] DB FLAVOR :POSTGRES
2026-08-12 16:53:30,392 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:53:30,392 [I] Checking connection..
2026-08-12 16:53:30,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:53:30,649 [I] Checking connection passed.
2026-08-12 16:53:30,649 [I] --------- Verifying version history table ---------
2026-08-12 16:53:30,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:53:30,914 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:53:30,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:53:31,167 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:53:31,168 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:53:31,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:53:31.403 UTC [30605] WARNING: there is no transaction in progress
2026-08-12 16:53:31,413 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:53:31,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:53:31,685 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:53:31,685 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:53:31,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:53:31,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:53:32,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-12 16:53:32,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:53:32,724 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:53:32,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:53:33,001 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:53:33,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:53:33,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:53:33,573 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:53:33,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:53:34.597 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.615 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.632 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.649 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.670 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.689 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.748 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.755 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.772 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.784 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.795 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.813 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.824 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.833 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.841 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:34.849 UTC [30820] WARNING: there is no transaction in progress
2026-08-12 16:53:35,553 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:53:35,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:53:35,791 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:53:35,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:53:36,064 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:53:36,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:53:36,317 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:53:36,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:53:36,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:53:36,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-12 16:53:37,107 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:53:37,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:53:37,385 [I] DB_PATCHES have already been applied
I20260812 16:53:37.392357 8974 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKFRmA3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:53:37.392462 8974 mini_ranger.cc:193] Using host: 127.8.195.148
I20260812 16:53:37.416009 8974 mini_ranger.cc:227] Ranger admin URL: http://127.8.195.148:34343
Aug 12, 2026 4:53:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:53:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:53:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:53:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:53:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:53:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:53:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:53:38.963354 8974 mini_ranger.cc:160] Time spent starting Ranger: real 9.496s user 0.000s sys 0.075s
Aug 12, 2026 4:53:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:53:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 4:53:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:53:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:54:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:54:01.408869 8974 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangerkms_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/rangerkms_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.195.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.195.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:54:01 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553641, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:54:01.660239 8974 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:54:01.690004 8974 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:54:02.556999 8974 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:54:02.557243 8974 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms
I20260812 16:54:02.557396 8974 mini_ranger_kms.cc:227] Using postgres at 127.8.195.148:51201
2026-08-12 16:54:03,057 [I] DB FLAVOR :POSTGRES
2026-08-12 16:54:03,058 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:54:03,058 [I] Checking connection
2026-08-12 16:54:03,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:54:03,300 [I] connection success
2026-08-12 16:54:03,300 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:54:03,300 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:54:03,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:54:03,543 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:54:03,543 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:54:03,544 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:54:03,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:51201/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:54:03,830 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:54:04.349924 8974 mini_ranger_kms.cc:310] Created kms service
I20260812 16:54:04.547290 8974 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:54:04.594623 8974 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:54:04 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553641, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:05.054093 8974 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:54:05.055389 8974 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:54:05.055467 8974 mini_ranger_kms.cc:240] Using host: 127.8.195.148
I20260812 16:54:05.083607 8974 mini_ranger_kms.cc:276] Ranger KMS PID: 31159
I20260812 16:54:05.083845 8974 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.195.148:37379
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:54:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 16:54:06.621551 8974 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.961s user 0.131s sys 0.404s
I20260812 16:54:06.621894 8974 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:54:06.622138 8974 mini_ranger_kms.cc:401] 127.8.195.148:37379/kms/v1/keys
Aug 12 16:54:07 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786553647, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:54:07 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553647, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
Aug 12 16:54:07 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553647, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
Aug 12 16:54:10 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:54:10.415170 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37869
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37869
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:37379/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:54:10.901506 31237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:54:10.901893 31237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:54:10.902064 31237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:54:10.910370 31237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:54:10.910495 31237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:54:10.910578 31237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:54:10.910727 31237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:54:10.910848 31237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:54:10.924777 31237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.195.148:37379/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37869
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:54:10.926262 31237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:54:10.927712 31237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:54:10.942744 31254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:10.942806 31249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:10.942858 31246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:54:10.943622 31237 server_base.cc:1034] running on GCE node
I20260812 16:54:10.944584 31237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:54:10.946300 31237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:54:10.947600 31237 hybrid_clock.cc:648] HybridClock initialized: now 1786553650947509 us; error 48 us; skew 500 ppm
I20260812 16:54:10.948036 31237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:54:10 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:54:10.951864 31237 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:54:10.953540 31237 webserver.cc:460] Webserver started at http://127.8.195.190:44131/ using document root <none> and password file <none>
I20260812 16:54:10.954267 31237 fs_manager.cc:359] Metadata directory not provided
I20260812 16:54:10.954419 31237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:54:10.954749 31237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:54:10 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:11.010571 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "be6090f7f664496b8de5a238fc8bf7d8"
format_stamp: "Formatted at 2026-08-12 16:54:11 on dist-test-slave-z8x9"
server_key: "5930af110122565d9ff6763c88760a9c"
server_key_iv: "391c92f7c0a00673e85ce40097df1600"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:11.011451 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "be6090f7f664496b8de5a238fc8bf7d8"
format_stamp: "Formatted at 2026-08-12 16:54:11 on dist-test-slave-z8x9"
server_key: "5930af110122565d9ff6763c88760a9c"
server_key_iv: "391c92f7c0a00673e85ce40097df1600"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:11.017634 31237 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 16:54:11.038415 31263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:11.040077 31237 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260812 16:54:11.040287 31237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "be6090f7f664496b8de5a238fc8bf7d8"
format_stamp: "Formatted at 2026-08-12 16:54:11 on dist-test-slave-z8x9"
server_key: "5930af110122565d9ff6763c88760a9c"
server_key_iv: "391c92f7c0a00673e85ce40097df1600"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:11.040539 31237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:54:11.079088 31237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:54:11.083760 31237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:54:11.084146 31237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:54:11.109838 31237 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37869
I20260812 16:54:11.110076 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37869 every 8 connection(s)
I20260812 16:54:11.111547 31237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:54:11.115494 31347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:54:11.120018 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 31237
I20260812 16:54:11.120252 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:54:11.121025 8974 external_mini_cluster.cc:1428] Setting key 731a853b2b087c77b5dc5c16a25c20b6
I20260812 16:54:11.128350 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: Bootstrap starting.
I20260812 16:54:11.132932 31347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: Neither blocks nor log segments found. Creating new log.
I20260812 16:54:11.134685 31347 log.cc:824] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:54:11.138530 31347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: No bootstrap required, opened a new log
Aug 12 16:54:11 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:54:11.150825 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:11.123503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.8.195.190:37869)
0812 16:54:11.125350 (+ 1847us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:11.125363 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:11.125432 (+ 69us) server_negotiation.cc:407] Connection header received
0812 16:54:11.127812 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:11.127837 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:11.128316 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:11.128786 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:11.130747 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:11.132410 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:11.134597 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:11.135893 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:11.136111 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:11.140857 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:11.140866 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:11.140886 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:11.140926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:11.146072 (+ 5146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:11.147059 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:11.147066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:11.147072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:11.147261 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:11.147724 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:11.147728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:11.147730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:11.148461 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:11.148769 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:11.149385 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 16:54:11.149785 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1441,"thread_start_us":1141,"threads_started":1}
I20260812 16:54:11.150615 31347 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } }
I20260812 16:54:11.151088 31347 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:54:11.151244 31347 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6090f7f664496b8de5a238fc8bf7d8, State: Initialized, Role: FOLLOWER
I20260812 16:54:11.151865 31347 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } }
I20260812 16:54:11.152187 31347 raft_consensus.cc:388] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:54:11.152351 31347 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:54:11.152573 31347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:54:11.155862 31347 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } }
I20260812 16:54:11.156476 31347 leader_election.cc:302] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be6090f7f664496b8de5a238fc8bf7d8; no voters:
I20260812 16:54:11.157259 31347 leader_election.cc:288] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:54:11.157441 31363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:54:11.158876 31363 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [term 1 LEADER]: Becoming Leader. State: Replica: be6090f7f664496b8de5a238fc8bf7d8, State: Running, Role: LEADER
I20260812 16:54:11.159546 31363 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } }
I20260812 16:54:11.160393 31347 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:54:11.162531 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be6090f7f664496b8de5a238fc8bf7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } } }
I20260812 16:54:11.163298 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:54:11.163964 31371 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6090f7f664496b8de5a238fc8bf7d8. Latest consensus state: current_term: 1 leader_uuid: "be6090f7f664496b8de5a238fc8bf7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6090f7f664496b8de5a238fc8bf7d8" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37869 } } }
I20260812 16:54:11.164834 31371 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8 [sys.catalog]: This master's current role is: LEADER
I20260812 16:54:11.169557 31375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:54:11.176476 31375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:54:11.185583 31347 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:54:11.188118 31375 catalog_manager.cc:1347] Generated new cluster ID: 692d7d29b0eb47a4b5676dcf2360298c
I20260812 16:54:11.188256 31375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:54:11.223690 31375 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:54:11.225383 31375 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:54:11.238986 31375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: Generated new TSK 0
I20260812 16:54:11.239984 31375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:54:12.073894 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:37379/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:54:12 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 16:54:12.539420 31430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:54:12.539839 31430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:54:12.540021 31430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:54:12.550822 31430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:54:12.551086 31430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:54:12.551295 31430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:54:12.566061 31430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.195.148:37379/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:54:12.568404 31430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:54:12.570551 31430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:54:12.595057 31448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:12.600805 31450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:54:12.602734 31430 server_base.cc:1034] running on GCE node
W20260812 16:54:12.603452 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:54:12.604226 31430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:54:12.605836 31430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:54:12.607239 31430 hybrid_clock.cc:648] HybridClock initialized: now 1786553652607065 us; error 73 us; skew 500 ppm
I20260812 16:54:12.607721 31430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:54:12 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:54:12.612051 31430 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:54:12.614578 31430 webserver.cc:460] Webserver started at http://127.8.195.129:38117/ using document root <none> and password file <none>
I20260812 16:54:12.615664 31430 fs_manager.cc:359] Metadata directory not provided
I20260812 16:54:12.615871 31430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:54:12.616297 31430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:54:12 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:12.637712 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "cb0ac632eb9a4145909c669546b9984d"
format_stamp: "Formatted at 2026-08-12 16:54:12 on dist-test-slave-z8x9"
server_key: "f459305813edf8b79ae0be81092d023a"
server_key_iv: "3d320c515c0f6ac1f08a1d8e8fc94aaa"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:12.639204 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "cb0ac632eb9a4145909c669546b9984d"
format_stamp: "Formatted at 2026-08-12 16:54:12 on dist-test-slave-z8x9"
server_key: "f459305813edf8b79ae0be81092d023a"
server_key_iv: "3d320c515c0f6ac1f08a1d8e8fc94aaa"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:12.668366 31430 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.012s sys 0.008s
I20260812 16:54:12.693540 31466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:12.695863 31430 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:54:12.696094 31430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "cb0ac632eb9a4145909c669546b9984d"
format_stamp: "Formatted at 2026-08-12 16:54:12 on dist-test-slave-z8x9"
server_key: "f459305813edf8b79ae0be81092d023a"
server_key_iv: "3d320c515c0f6ac1f08a1d8e8fc94aaa"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:12.696399 31430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:54:12.755858 31430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:54:12.764077 31430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:54:12.764567 31430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:54:12.766157 31430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:54:12.768815 31430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:54:12.768916 31430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:12.769069 31430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:54:12.769150 31430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:12.881738 31430 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:42663
I20260812 16:54:12.881765 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:42663 every 8 connection(s)
I20260812 16:54:12.883631 31430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:54:12.893270 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 31430
I20260812 16:54:12.893493 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:54:12.894634 8974 external_mini_cluster.cc:1428] Setting key de731a7239c7d29db0ca94ab23072810
Aug 12 16:54:12 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
I20260812 16:54:12.944423 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:12.887076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:38309 (local address 127.8.195.190:37869)
0812 16:54:12.903668 (+ 16592us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:12.903677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:12.906972 (+ 3295us) server_negotiation.cc:407] Connection header received
0812 16:54:12.915986 (+ 9014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:12.915994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:12.916100 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:12.916191 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:12.918887 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:12.920274 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:12.923235 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:12.924013 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:12.924113 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:12.929802 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:12.929812 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:12.929819 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:12.929882 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:12.934461 (+ 4579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:12.937255 (+ 2794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:12.937263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:12.937267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:12.937477 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:12.938293 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:12.938299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:12.938304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:12.940176 (+ 1872us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:12.940371 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:12.943589 (+ 3218us) server_negotiation.cc:299] Negotiation successful
0812 16:54:12.943914 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16424,"thread_start_us":16274,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
I20260812 16:54:12.947312 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:12.887931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37869 (local address 127.8.195.129:38309)
0812 16:54:12.905400 (+ 17469us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:12.905473 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:12.915581 (+ 10108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:12.916837 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:12.916851 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:12.917687 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:54:12.918655 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:12.918684 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:12.920588 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:12.920593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:12.923041 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:12.923058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:12.924570 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:12.924577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:12.924958 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:12.925877 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:54:12.925908 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:54:12.929399 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:54:12.936209 (+ 6810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:12.936224 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:12.936248 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:12.937040 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:12.937658 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:12.937664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:12.937683 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:12.938116 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:12.940503 (+ 2387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:54:12.940512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:54:12.941076 (+ 564us) client_negotiation.cc:769] Sending connection context
0812 16:54:12.944871 (+ 3795us) client_negotiation.cc:240] Negotiation successful
0812 16:54:12.945629 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17011,"thread_start_us":18024,"threads_started":1}
I20260812 16:54:12.951373 31669 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37869
I20260812 16:54:12.951751 31669 heartbeater.cc:464] Registering TS with master...
I20260812 16:54:12.952677 31669 heartbeater.cc:511] Master 127.8.195.190:37869 requested a full tablet report, sending...
I20260812 16:54:12.955513 31288 ts_manager.cc:194] Registered new tserver with Master: cb0ac632eb9a4145909c669546b9984d (127.8.195.129:42663)
I20260812 16:54:12.958009 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:38309
Aug 12 16:54:12 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:54:12.993813 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:37379/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:54:13 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553652, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
W20260812 16:54:13.604897 31680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:54:13.605399 31680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:54:13.605679 31680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:54:13.617303 31680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:54:13.617457 31680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:54:13.617640 31680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:54:13.633633 31680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.195.148:37379/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:54:13.635514 31680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:54:13.637547 31680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:54:13.658006 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:13.658030 31707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:13.662658 31713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:54:13.667080 31680 server_base.cc:1034] running on GCE node
I20260812 16:54:13.667848 31680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:54:13.669137 31680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:54:13.670481 31680 hybrid_clock.cc:648] HybridClock initialized: now 1786553653670419 us; error 55 us; skew 500 ppm
I20260812 16:54:13.670910 31680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:54:13 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553653, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:54:13.675388 31680 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:54:13.677866 31680 webserver.cc:460] Webserver started at http://127.8.195.130:45493/ using document root <none> and password file <none>
I20260812 16:54:13.678745 31680 fs_manager.cc:359] Metadata directory not provided
I20260812 16:54:13.678923 31680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:54:13.679405 31680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:54:13 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553653, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:13.699522 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "e902fe381f3c48bdbea44db4158997fe"
format_stamp: "Formatted at 2026-08-12 16:54:13 on dist-test-slave-z8x9"
server_key: "5009caa24cce5c107455be3cd0b887d2"
server_key_iv: "9395e593e63513fc25e5b8f1ca2c0798"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:13.700949 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "e902fe381f3c48bdbea44db4158997fe"
format_stamp: "Formatted at 2026-08-12 16:54:13 on dist-test-slave-z8x9"
server_key: "5009caa24cce5c107455be3cd0b887d2"
server_key_iv: "9395e593e63513fc25e5b8f1ca2c0798"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:13.707980 31680 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 16:54:13.728401 31722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:13.730605 31680 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 16:54:13.730831 31680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "e902fe381f3c48bdbea44db4158997fe"
format_stamp: "Formatted at 2026-08-12 16:54:13 on dist-test-slave-z8x9"
server_key: "5009caa24cce5c107455be3cd0b887d2"
server_key_iv: "9395e593e63513fc25e5b8f1ca2c0798"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:13.731117 31680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:54:13.764853 31680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:54:13.771356 31680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:54:13.771763 31680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:54:13.773546 31680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:54:13.775811 31680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:54:13.775956 31680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 16:54:13.776115 31680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:54:13.776211 31680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:13.836652 31680 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:43987
I20260812 16:54:13.836750 31935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:43987 every 8 connection(s)
I20260812 16:54:13.838538 31680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:54:13.840260 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 31680
I20260812 16:54:13.840448 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:54:13.841001 8974 external_mini_cluster.cc:1428] Setting key 7a23e08866e4763a5e7f9416fa92adf8
Aug 12 16:54:13 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553653, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:54:13.873137 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:13.843868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:45925 (local address 127.8.195.190:37869)
0812 16:54:13.853764 (+ 9896us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:13.853783 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:13.853814 (+ 31us) server_negotiation.cc:407] Connection header received
0812 16:54:13.853893 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:13.853900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:13.854014 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:13.854161 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:13.856789 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:13.858075 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:13.860112 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:13.860551 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:13.860748 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:13.865213 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:13.865227 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:13.865231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:13.865270 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:13.869408 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:13.870363 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:13.870368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:13.870370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:13.870507 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:13.871037 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:13.871041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:13.871043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:13.871514 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:13.871664 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:13.872484 (+ 820us) server_negotiation.cc:299] Negotiation successful
0812 16:54:13.872724 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9733,"thread_start_us":9562,"threads_started":1}
I20260812 16:54:13.874526 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:13.842818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37869 (local address 127.8.195.130:45925)
0812 16:54:13.843944 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:13.844001 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:13.846908 (+ 2907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:13.854661 (+ 7753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:13.854676 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:13.855399 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:54:13.856538 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:13.856569 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:13.858291 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:13.858296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:13.859944 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:13.859959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:13.860963 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:13.860974 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:13.861310 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:13.862547 (+ 1237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:54:13.862573 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:54:13.864933 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:54:13.869586 (+ 4653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:13.869605 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:13.869620 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:13.870173 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:13.870641 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:13.870647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:13.870650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:13.870879 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:13.871656 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:54:13.871665 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:54:13.872037 (+ 372us) client_negotiation.cc:769] Sending connection context
0812 16:54:13.872378 (+ 341us) client_negotiation.cc:240] Negotiation successful
0812 16:54:13.872987 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":1162,"threads_started":1}
I20260812 16:54:13.876133 31936 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37869
I20260812 16:54:13.876571 31936 heartbeater.cc:464] Registering TS with master...
I20260812 16:54:13.877482 31936 heartbeater.cc:511] Master 127.8.195.190:37869 requested a full tablet report, sending...
I20260812 16:54:13.879221 31288 ts_manager.cc:194] Registered new tserver with Master: e902fe381f3c48bdbea44db4158997fe (127.8.195.130:43987)
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
I20260812 16:54:13.880412 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:45925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:54:13.921027 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:37379/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:54:13.963429 31669 heartbeater.cc:503] Master 127.8.195.190:37869 was elected leader, sending a full tablet report...
W20260812 16:54:14.358767 31944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:54:14.359134 31944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:54:14.359371 31944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:54:14.367352 31944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:54:14.367520 31944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:54:14.367736 31944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:54:14.379174 31944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.195.148:37379/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:54:14.380599 31944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:54:14.382037 31944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:54:14.397699 31953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:14.397811 31956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:54:14.397732 31954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:54:14.399191 31944 server_base.cc:1034] running on GCE node
I20260812 16:54:14.399955 31944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:54:14.400990 31944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:54:14.402240 31944 hybrid_clock.cc:648] HybridClock initialized: now 1786553654402164 us; error 65 us; skew 500 ppm
I20260812 16:54:14.402616 31944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:54:14 dist-test-slave-z8x9 krb5kdc[30406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553654, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:54:14.406425 31944 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:54:14.408229 31944 webserver.cc:460] Webserver started at http://127.8.195.131:36529/ using document root <none> and password file <none>
I20260812 16:54:14.409021 31944 fs_manager.cc:359] Metadata directory not provided
I20260812 16:54:14.409267 31944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:54:14.409660 31944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:54:14 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553654, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:14.425115 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "8c506b91d63f4fa4964a5572d6a40afc"
format_stamp: "Formatted at 2026-08-12 16:54:14 on dist-test-slave-z8x9"
server_key: "d02fc65fcd6e57c62ea997073b71ce78"
server_key_iv: "c179ccd91eb28b9e7f6e877d2eeac3ae"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:14.426102 31944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "8c506b91d63f4fa4964a5572d6a40afc"
format_stamp: "Formatted at 2026-08-12 16:54:14 on dist-test-slave-z8x9"
server_key: "d02fc65fcd6e57c62ea997073b71ce78"
server_key_iv: "c179ccd91eb28b9e7f6e877d2eeac3ae"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:14.432313 31944 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260812 16:54:14.450771 31966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:14.452535 31944 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 16:54:14.452780 31944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "8c506b91d63f4fa4964a5572d6a40afc"
format_stamp: "Formatted at 2026-08-12 16:54:14 on dist-test-slave-z8x9"
server_key: "d02fc65fcd6e57c62ea997073b71ce78"
server_key_iv: "c179ccd91eb28b9e7f6e877d2eeac3ae"
server_key_version: "kuduclusterkey@0"
I20260812 16:54:14.453087 31944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:54:14.474205 31944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:54:14.479008 31944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:54:14.479344 31944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:54:14.480450 31944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:54:14.482265 31944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:54:14.482407 31944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:14.482582 31944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:54:14.482661 31944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:54:14.534790 31944 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:45499
I20260812 16:54:14.534792 32156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:45499 every 8 connection(s)
I20260812 16:54:14.536346 31944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:54:14.544945 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 31944
I20260812 16:54:14.545212 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:54:14.545739 8974 external_mini_cluster.cc:1428] Setting key fa05ec75e7447dec0483bd2d115be452
Aug 12 16:54:14 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553654, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:54:14.561088 32161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.539586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:32991 (local address 127.8.195.190:37869)
0812 16:54:14.540465 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.540474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.541059 (+ 585us) server_negotiation.cc:407] Connection header received
0812 16:54:14.543347 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.543353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.543470 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.543596 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:14.545577 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.546921 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.549469 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.549974 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.550075 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.553999 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:14.554005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:14.554008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:14.554043 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:14.557606 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.558437 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.558441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.558444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.558627 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.559096 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.559099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.559101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.559374 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:14.559484 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.560419 (+ 935us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.560698 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":525,"threads_started":1}
I20260812 16:54:14.562258 32162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.540032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37869 (local address 127.8.195.131:32991)
0812 16:54:14.540875 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.540915 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.543041 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.543890 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.543899 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.544476 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:54:14.545392 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.545420 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.547120 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.547126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.549227 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.549244 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.550191 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.550196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.550407 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.551072 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:54:14.551096 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:54:14.553752 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:54:14.557738 (+ 3986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:14.557745 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:14.557757 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:14.558318 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:14.558723 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:54:14.558727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:54:14.558729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:54:14.558991 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:54:14.559504 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:54:14.559512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:54:14.559952 (+ 440us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.560294 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.560881 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":400,"threads_started":1}
I20260812 16:54:14.563589 32158 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37869
I20260812 16:54:14.563915 32158 heartbeater.cc:464] Registering TS with master...
I20260812 16:54:14.564656 32158 heartbeater.cc:511] Master 127.8.195.190:37869 requested a full tablet report, sending...
I20260812 16:54:14.566237 31288 ts_manager.cc:194] Registered new tserver with Master: 8c506b91d63f4fa4964a5572d6a40afc (127.8.195.131:45499)
I20260812 16:54:14.567157 31288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:32991
I20260812 16:54:14.574381 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:54:14.596063 32161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.579799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.8.195.190:37869)
0812 16:54:14.579967 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.579972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.580668 (+ 696us) server_negotiation.cc:407] Connection header received
0812 16:54:14.580893 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.580897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.581017 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.581126 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:14.583007 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.583987 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.586195 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.586948 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.587101 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.590274 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:14.590280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:14.590283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:14.590316 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:14.593493 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.594237 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.594242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.594246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.594375 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.594709 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.594712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.594714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.595040 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:14.595219 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.595623 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.595822 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:54:14.600538 31288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:54:14.608991 31288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:54:14.647675 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.623518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35540)
0812 16:54:14.636757 (+ 13239us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.636823 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.637151 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.639107 (+ 1956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.639116 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.639148 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:54:14.639511 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.639520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.641586 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.641592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.644398 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.644411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.646496 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.646504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.646840 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.646899 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.647064 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.647257 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12864,"mutex_wait_us":19,"thread_start_us":14889,"threads_started":1}
I20260812 16:54:14.647644 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.624443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.8.195.129:42663)
0812 16:54:14.637540 (+ 13097us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.637552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.637596 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:54:14.638474 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.638485 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.638792 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.638975 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:54:14.639685 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.641412 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.644567 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.646211 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.646417 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.646789 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.647112 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.647285 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12876,"thread_start_us":12590,"threads_started":1}
I20260812 16:54:14.649148 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.625045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.8.195.130:43987)
0812 16:54:14.636744 (+ 11699us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.636755 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.638889 (+ 2134us) server_negotiation.cc:407] Connection header received
0812 16:54:14.639766 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.639771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.639930 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.640130 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:54:14.641086 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.642924 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.646344 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.648096 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.648252 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.648457 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.648703 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.648823 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11550,"thread_start_us":11416,"threads_started":1}
I20260812 16:54:14.650558 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.623676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45499 (local address 127.0.0.1:43684)
0812 16:54:14.638417 (+ 14741us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.638441 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.638745 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.639760 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.639767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.639793 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:54:14.640330 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.640343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.642625 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.642628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.645761 (+ 3133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.645776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.649575 (+ 3799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.649586 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.649854 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.649886 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.650006 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.650208 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14445,"thread_start_us":14214,"threads_started":1}
I20260812 16:54:14.650790 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.624110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.8.195.131:45499)
0812 16:54:14.636216 (+ 12106us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.636229 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.638584 (+ 2355us) server_negotiation.cc:407] Connection header received
0812 16:54:14.639038 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.639047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.639275 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.639435 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:54:14.640575 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.642235 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.645943 (+ 3708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.647561 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.647715 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.648318 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.650094 (+ 1776us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.650269 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11853,"thread_start_us":11632,"threads_started":1}
I20260812 16:54:14.651078 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.623983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:43987 (local address 127.0.0.1:39342)
0812 16:54:14.638736 (+ 14753us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.638757 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.638886 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.640131 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.640136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.640166 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:54:14.640838 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.640855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.643123 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.643130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.646101 (+ 2971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.646167 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.648336 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.648344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.648510 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.648541 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.648649 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.648814 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14615,"mutex_wait_us":1,"thread_start_us":13209,"threads_started":1}
I20260812 16:54:14.652983 31561 tablet_service.cc:1467] Processing CreateTablet for tablet 99574b89dad94f8fb47bf72234cfd536 (DEFAULT_TABLE table=test-table [id=336c14d843094d33a6b4da9ce1c6d563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:54:14.654352 31814 tablet_service.cc:1467] Processing CreateTablet for tablet 99574b89dad94f8fb47bf72234cfd536 (DEFAULT_TABLE table=test-table [id=336c14d843094d33a6b4da9ce1c6d563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:54:14.654850 31561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99574b89dad94f8fb47bf72234cfd536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:54:14.654632 32052 tablet_service.cc:1467] Processing CreateTablet for tablet 99574b89dad94f8fb47bf72234cfd536 (DEFAULT_TABLE table=test-table [id=336c14d843094d33a6b4da9ce1c6d563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:54:14.655570 31814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99574b89dad94f8fb47bf72234cfd536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:54:14.656220 32052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99574b89dad94f8fb47bf72234cfd536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:54:14.672380 32192 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Bootstrap starting.
I20260812 16:54:14.677937 32192 tablet_bootstrap.cc:654] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Neither blocks nor log segments found. Creating new log.
I20260812 16:54:14.678591 32197 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Bootstrap starting.
I20260812 16:54:14.678685 32199 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Bootstrap starting.
I20260812 16:54:14.680016 32192 log.cc:824] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Log is configured to *not* fsync() on all Append() calls
I20260812 16:54:14.682535 32197 tablet_bootstrap.cc:654] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Neither blocks nor log segments found. Creating new log.
I20260812 16:54:14.683053 32199 tablet_bootstrap.cc:654] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Neither blocks nor log segments found. Creating new log.
I20260812 16:54:14.683691 32197 log.cc:824] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:54:14.684633 32199 log.cc:824] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:54:14.686445 32192 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: No bootstrap required, opened a new log
I20260812 16:54:14.686944 32192 ts_tablet_manager.cc:1397] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260812 16:54:14.688285 32197 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: No bootstrap required, opened a new log
I20260812 16:54:14.688308 32199 tablet_bootstrap.cc:492] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: No bootstrap required, opened a new log
I20260812 16:54:14.688591 32199 ts_tablet_manager.cc:1397] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260812 16:54:14.688589 32197 ts_tablet_manager.cc:1397] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 16:54:14.697584 32199 raft_consensus.cc:348] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.698030 32199 raft_consensus.cc:374] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:54:14.698204 32199 raft_consensus.cc:729] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c506b91d63f4fa4964a5572d6a40afc, State: Initialized, Role: FOLLOWER
I20260812 16:54:14.699004 32199 consensus_queue.cc:260] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.700353 32158 heartbeater.cc:503] Master 127.8.195.190:37869 was elected leader, sending a full tablet report...
I20260812 16:54:14.700316 32192 raft_consensus.cc:348] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.700778 32192 raft_consensus.cc:374] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:54:14.700881 32192 raft_consensus.cc:729] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e902fe381f3c48bdbea44db4158997fe, State: Initialized, Role: FOLLOWER
I20260812 16:54:14.701273 32199 ts_tablet_manager.cc:1428] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Time spent starting tablet: real 0.012s user 0.004s sys 0.009s
I20260812 16:54:14.701059 32197 raft_consensus.cc:348] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.701576 32197 raft_consensus.cc:374] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:54:14.701676 32197 raft_consensus.cc:729] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0ac632eb9a4145909c669546b9984d, State: Initialized, Role: FOLLOWER
I20260812 16:54:14.701768 32192 consensus_queue.cc:260] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.702632 32197 consensus_queue.cc:260] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.703316 31936 heartbeater.cc:503] Master 127.8.195.190:37869 was elected leader, sending a full tablet report...
I20260812 16:54:14.704109 32192 ts_tablet_manager.cc:1428] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260812 16:54:14.706022 32197 ts_tablet_manager.cc:1428] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260812 16:54:14.777004 32211 raft_consensus.cc:482] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:54:14.777374 32211 raft_consensus.cc:504] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.779361 32211 leader_election.cc:288] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c506b91d63f4fa4964a5572d6a40afc (127.8.195.131:45499), e902fe381f3c48bdbea44db4158997fe (127.8.195.130:43987)
W20260812 16:54:14.788828 32159 tablet.cc:2369] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:54:14.789582 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.779702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:43987 (local address 127.8.195.129:55117)
0812 16:54:14.780645 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.780731 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.780917 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.781324 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.781328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.781359 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:54:14.781823 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.781833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.783000 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.783006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.785263 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.785282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.787248 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.787260 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.787532 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.787586 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.788783 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.788937 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":132,"thread_start_us":546,"threads_started":1}
I20260812 16:54:14.788329 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.779898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:55117 (local address 127.8.195.130:43987)
0812 16:54:14.780167 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.780217 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.780838 (+ 621us) server_negotiation.cc:407] Connection header received
0812 16:54:14.781077 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.781082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.781141 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.781291 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:54:14.781977 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.782858 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.785443 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.786849 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.787006 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.787085 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.787799 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.787973 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:54:14.790934 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.779765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34935 (local address 127.8.195.131:45499)
0812 16:54:14.779992 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.779999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.781490 (+ 1491us) server_negotiation.cc:407] Connection header received
0812 16:54:14.781714 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.781723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.781783 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.781897 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:54:14.782635 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.783450 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.786819 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.788409 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.788595 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.788710 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.790445 (+ 1735us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.790592 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:54:14.791826 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.780152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:45499 (local address 127.8.195.129:34935)
0812 16:54:14.781363 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:14.781378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:54:14.781561 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:54:14.781915 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:54:14.781920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:54:14.781942 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:54:14.782464 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.782475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.783617 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.783622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.786613 (+ 2991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:54:14.786630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.788773 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.788780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:54:14.790208 (+ 1428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.790264 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 16:54:14.791128 (+ 864us) client_negotiation.cc:240] Negotiation successful
0812 16:54:14.791286 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"mutex_wait_us":109,"thread_start_us":777,"threads_started":1}
I20260812 16:54:14.791668 31854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99574b89dad94f8fb47bf72234cfd536" candidate_uuid: "cb0ac632eb9a4145909c669546b9984d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e902fe381f3c48bdbea44db4158997fe" is_pre_election: true
I20260812 16:54:14.792411 31854 raft_consensus.cc:2454] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0ac632eb9a4145909c669546b9984d in term 0.
I20260812 16:54:14.793121 32084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99574b89dad94f8fb47bf72234cfd536" candidate_uuid: "cb0ac632eb9a4145909c669546b9984d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c506b91d63f4fa4964a5572d6a40afc" is_pre_election: true
I20260812 16:54:14.793363 31468 leader_election.cc:302] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0ac632eb9a4145909c669546b9984d, e902fe381f3c48bdbea44db4158997fe; no voters:
I20260812 16:54:14.793910 32211 raft_consensus.cc:2781] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:54:14.793823 32084 raft_consensus.cc:2454] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0ac632eb9a4145909c669546b9984d in term 0.
I20260812 16:54:14.794185 32211 raft_consensus.cc:482] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:54:14.794396 32211 raft_consensus.cc:3037] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:54:14.798225 32211 raft_consensus.cc:504] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.799417 32211 leader_election.cc:288] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [CANDIDATE]: Term 1 election: Requested vote from peers 8c506b91d63f4fa4964a5572d6a40afc (127.8.195.131:45499), e902fe381f3c48bdbea44db4158997fe (127.8.195.130:43987)
I20260812 16:54:14.799393 32084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99574b89dad94f8fb47bf72234cfd536" candidate_uuid: "cb0ac632eb9a4145909c669546b9984d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c506b91d63f4fa4964a5572d6a40afc"
I20260812 16:54:14.799603 32084 raft_consensus.cc:3037] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:54:14.799849 31854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99574b89dad94f8fb47bf72234cfd536" candidate_uuid: "cb0ac632eb9a4145909c669546b9984d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e902fe381f3c48bdbea44db4158997fe"
I20260812 16:54:14.800091 31854 raft_consensus.cc:3037] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:54:14.802645 32084 raft_consensus.cc:2454] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0ac632eb9a4145909c669546b9984d in term 1.
I20260812 16:54:14.803881 31854 raft_consensus.cc:2454] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0ac632eb9a4145909c669546b9984d in term 1.
I20260812 16:54:14.804396 31472 leader_election.cc:302] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c506b91d63f4fa4964a5572d6a40afc, cb0ac632eb9a4145909c669546b9984d; no voters:
I20260812 16:54:14.805308 32214 raft_consensus.cc:2781] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:54:14.805563 32214 raft_consensus.cc:686] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 1 LEADER]: Becoming Leader. State: Replica: cb0ac632eb9a4145909c669546b9984d, State: Running, Role: LEADER
I20260812 16:54:14.806279 32214 consensus_queue.cc:237] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } }
I20260812 16:54:14.813352 31286 catalog_manager.cc:5495] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0ac632eb9a4145909c669546b9984d (127.8.195.129). New cstate: current_term: 1 leader_uuid: "cb0ac632eb9a4145909c669546b9984d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0ac632eb9a4145909c669546b9984d" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 } health_report { overall_health: UNKNOWN } } }
W20260812 16:54:14.843604 31937 tablet.cc:2369] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:54:14.853024 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.846891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.8.195.129:42663)
0812 16:54:14.847156 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.847163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.847205 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:54:14.847467 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.847473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.847581 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.847719 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:54:14.848092 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.849209 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.850684 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.851289 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.851417 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.851925 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:54:14.852425 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:54:14.852541 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.852650 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.852718 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:54:14.862573 31854 raft_consensus.cc:1261] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 1 FOLLOWER]: Refusing update from remote peer cb0ac632eb9a4145909c669546b9984d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:54:14.862614 32084 raft_consensus.cc:1261] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 1 FOLLOWER]: Refusing update from remote peer cb0ac632eb9a4145909c669546b9984d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:54:14.863657 32214 consensus_queue.cc:1035] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c506b91d63f4fa4964a5572d6a40afc" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 45499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:54:14.864070 32211 consensus_queue.cc:1035] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e902fe381f3c48bdbea44db4158997fe" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 43987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:54:14.876448 32221 mvcc.cc:205] Tried to move back new op lower bound from 7317723770305708032 to 7317723770091884544. Current Snapshot: MvccSnapshot[applied={T|T < 7317723770305708032}]
I20260812 16:54:14.902922 31286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:54:14.903282 31286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:54:14.907073 31286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be6090f7f664496b8de5a238fc8bf7d8: Sending DeleteTablet for 3 replicas of tablet 99574b89dad94f8fb47bf72234cfd536
I20260812 16:54:14.908604 31561 tablet_service.cc:1514] Processing DeleteTablet for tablet 99574b89dad94f8fb47bf72234cfd536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:54:14 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35540
I20260812 16:54:14.908783 32052 tablet_service.cc:1514] Processing DeleteTablet for tablet 99574b89dad94f8fb47bf72234cfd536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:54:14 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:43684
I20260812 16:54:14.909497 31814 tablet_service.cc:1514] Processing DeleteTablet for tablet 99574b89dad94f8fb47bf72234cfd536 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:54:14 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:39342
I20260812 16:54:14.910735 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 31430
I20260812 16:54:14.930270 32237 tablet_replica.cc:330] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: stopping tablet replica
I20260812 16:54:14.931066 32239 tablet_replica.cc:330] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: stopping tablet replica
I20260812 16:54:14.931552 32239 raft_consensus.cc:2229] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:54:14.932224 32239 raft_consensus.cc:2258] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:54:14.931030 32237 raft_consensus.cc:2229] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:54:14.933729 32237 raft_consensus.cc:2258] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:54:14.936668 32237 ts_tablet_manager.cc:1905] T 99574b89dad94f8fb47bf72234cfd536 P cb0ac632eb9a4145909c669546b9984d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 16:54:14 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:54:14.937016 32238 tablet_replica.cc:330] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: stopping tablet replica
I20260812 16:54:14.937505 32238 raft_consensus.cc:2229] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:54:14.938200 32238 raft_consensus.cc:2258] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:54:14.940552 32238 ts_tablet_manager.cc:1905] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:54:14.940789 32239 ts_tablet_manager.cc:1905] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:54:14.948676 32238 ts_tablet_manager.cc:1918] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:54:14.948940 32238 log.cc:1196] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/99574b89dad94f8fb47bf72234cfd536
I20260812 16:54:14.949152 32239 ts_tablet_manager.cc:1918] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:54:14.949429 32239 log.cc:1196] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/99574b89dad94f8fb47bf72234cfd536
I20260812 16:54:14.950007 32239 ts_tablet_manager.cc:1939] T 99574b89dad94f8fb47bf72234cfd536 P e902fe381f3c48bdbea44db4158997fe: Deleting consensus metadata
I20260812 16:54:14.950031 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:14.911981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.8.195.129:42663)
0812 16:54:14.912191 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:14.912198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:14.922358 (+ 10160us) server_negotiation.cc:407] Connection header received
0812 16:54:14.922903 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:14.922912 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:14.923016 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:14.923106 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:14.925501 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.927068 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.930204 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:14.931535 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:14.931721 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:14.939902 (+ 8181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:14.939918 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:14.939935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:14.939980 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:14.944589 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.946315 (+ 1726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.946325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.946331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.946533 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:14.947021 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:14.947026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:14.947030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:14.947545 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:14.947949 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:14.949238 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0812 16:54:14.949498 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:54:14.950361 32238 ts_tablet_manager.cc:1939] T 99574b89dad94f8fb47bf72234cfd536 P 8c506b91d63f4fa4964a5572d6a40afc: Deleting consensus metadata
I20260812 16:54:14.950886 31662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:54:14.952042 31264 catalog_manager.cc:4841] TS e902fe381f3c48bdbea44db4158997fe (127.8.195.130:43987): tablet 99574b89dad94f8fb47bf72234cfd536 (table test-table [id=336c14d843094d33a6b4da9ce1c6d563]) successfully deleted
I20260812 16:54:14.954937 31270 catalog_manager.cc:4841] TS 8c506b91d63f4fa4964a5572d6a40afc (127.8.195.131:45499): tablet 99574b89dad94f8fb47bf72234cfd536 (table test-table [id=336c14d843094d33a6b4da9ce1c6d563]) successfully deleted
I20260812 16:54:15.056417 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 31430
I20260812 16:54:15.082511 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 31680
W20260812 16:54:15.082501 31266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:42663 (error 108)
I20260812 16:54:15.085414 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.084813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35568)
0812 16:54:15.084981 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.085119 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:54:15.085638 31266 catalog_manager.cc:4584] TS cb0ac632eb9a4145909c669546b9984d (127.8.195.129:42663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99574b89dad94f8fb47bf72234cfd536: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Aug 12 16:54:15 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:54:15.099378 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.083453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.8.195.130:43987)
0812 16:54:15.083641 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:15.083647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:15.083754 (+ 107us) server_negotiation.cc:407] Connection header received
0812 16:54:15.083973 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:15.083983 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:15.084056 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:15.084168 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:15.085674 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:15.086893 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:15.088370 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:15.088792 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:15.088899 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:15.093321 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:15.093337 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:15.093353 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:15.093387 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:15.096732 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:15.097471 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:15.097479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:15.097481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:15.097611 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:15.097983 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:15.097987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:15.097989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:15.098354 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:15.098587 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:15.098829 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 16:54:15.099007 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:54:15.100128 31928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:54:15.139475 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.138977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35570)
0812 16:54:15.139198 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.139312 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:54:15.140580 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.140191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35586)
0812 16:54:15.140375 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.140466 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 16:54:15.192914 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 31680
I20260812 16:54:15.202065 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.201437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35600)
0812 16:54:15.201687 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.201825 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:54:15.204345 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.203885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35602)
0812 16:54:15.204070 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.204174 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:54:15.223501 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 31944
Aug 12 16:54:15 dist-test-slave-z8x9 krb5kdc[30406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:54:15.239265 32189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.224437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.8.195.131:45499)
0812 16:54:15.224629 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 16:54:15.224643 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:54:15.224689 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:54:15.224765 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:54:15.224769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:54:15.224836 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:54:15.224924 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:54:15.226417 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:15.227161 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:15.228460 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:54:15.228846 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:54:15.228941 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:54:15.233824 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:54:15.233840 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:54:15.233857 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:54:15.233892 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:54:15.236683 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:15.237440 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:15.237448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:15.237450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:15.237589 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:54:15.237934 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:54:15.237937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:54:15.237939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:54:15.238327 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:54:15.238544 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:54:15.238828 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 16:54:15.238997 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:54:15.239910 32150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:54:15.285063 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.284454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35618)
0812 16:54:15.284700 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.284829 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 16:54:15.286271 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.285878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35630)
0812 16:54:15.286074 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.286151 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260812 16:54:15.332005 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 31944
I20260812 16:54:15.357968 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 31237
I20260812 16:54:15.359227 31342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:54:15.475867 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:54:15.475235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42663 (local address 127.0.0.1:35636)
0812 16:54:15.475451 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 16:54:15.475588 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 16:54:15.475944 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 31237
I20260812 16:54:15.516206 8974 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:54:15.861994 8974 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:54:15.862345 8974 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:54:16.208673 8974 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:54:16.209 UTC [30457] LOG: received smart shutdown request
2026-08-12 16:54:16.213 UTC [30457] LOG: background worker "logical replication launcher" (PID 30479) exited with exit code 1
2026-08-12 16:54:16.213 UTC [30465] LOG: shutting down
2026-08-12 16:54:16.386 UTC [30457] LOG: database system is shut down
2026-08-12T16:54:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53101 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:54:16 dist-test-slave-z8x9 krb5kdc[32258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:54:16 dist-test-slave-z8x9 krb5kdc[32258](info): set up 2 sockets
Aug 12 16:54:16 dist-test-slave-z8x9 krb5kdc[32258](info): commencing operation
krb5kdc: starting...
W20260812 16:54:17.955200 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:54:18 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:54:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:54:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangeradmin_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangeradmin_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangerlookup_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangerlookup_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
I20260812 16:54:18.388644 8974 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKFRmA3/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/postgres -l logfile start
2026-08-12 16:54:21.106 UTC [32310] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 16:54:21.106 UTC [32310] LOG: listening on IPv4 address "127.8.195.148", port 60929
2026-08-12 16:54:21.109 UTC [32310] LOG: listening on Unix socket "/tmp/.s.PGSQL.60929"
2026-08-12 16:54:21.134 UTC [32315] LOG: database system was shut down at 2026-08-12 16:54:19 UTC
2026-08-12 16:54:21.138 UTC [32310] LOG: database system is ready to accept connections
I20260812 16:54:22.410991 8974 mini_postgres.cc:96] Postgres bound to 60929
2026-08-12 16:54:22.436 UTC [32330] FATAL: database "slave" does not exist
127.8.195.148:60929 - accepting connections
I20260812 16:54:22.437299 8974 mini_ranger.cc:161] Starting Ranger...
I20260812 16:54:22.470681 8974 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 16:54:23.021255 8974 mini_ranger.cc:87] Created ranger Postgres database
I20260812 16:54:23.021397 8974 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin
2026-08-12 16:54:23,329 [I] DB FLAVOR :POSTGRES
2026-08-12 16:54:23,330 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:54:23,330 [I] Checking connection..
2026-08-12 16:54:23,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 16:54:23,598 [I] Checking connection passed.
2026-08-12 16:54:23,598 [I] --------- Verifying version history table ---------
2026-08-12 16:54:23,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:54:23,842 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:54:23,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:54:24,118 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 16:54:24,118 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 16:54:24,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 16:54:24.386 UTC [32436] WARNING: there is no transaction in progress
2026-08-12 16:54:24,396 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 16:54:24,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 16:54:24,672 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 16:54:24,672 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:54:24,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 16:54:24,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 16:54:25,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-12 16:54:25,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:54:25,716 [I] Table x_portal_user does not exist in database ranger
2026-08-12 16:54:25,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:54:25,990 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 16:54:25,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:54:26,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:54:26,501 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 16:54:26,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 16:54:27.508 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.524 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.540 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.557 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.574 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.592 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.651 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.659 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.674 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.685 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.696 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.711 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.721 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.728 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.735 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:27.743 UTC [32635] WARNING: there is no transaction in progress
2026-08-12 16:54:28,421 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 16:54:28,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 16:54:28,680 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 16:54:28,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 16:54:28,960 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 16:54:28,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 16:54:29,240 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 16:54:29,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 16:54:29,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 16:54:29,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-12 16:54:29,992 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 16:54:29,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 16:54:30,266 [I] DB_PATCHES have already been applied
I20260812 16:54:30.272562 8974 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKFRmA3/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 16:54:30.272665 8974 mini_ranger.cc:193] Using host: 127.8.195.148
I20260812 16:54:30.293994 8974 mini_ranger.cc:227] Ranger admin URL: http://127.8.195.148:40901
Aug 12, 2026 4:54:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:54:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:54:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 4:54:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 4:54:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 4:54:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 4:54:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 16:54:31.814224 8974 mini_ranger.cc:160] Time spent starting Ranger: real 9.377s user 0.000s sys 0.062s
Aug 12, 2026 4:54:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 4:54:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 4:54:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 4:54:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 4:54:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 16:54:54.360343 8974 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.195.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.195.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangerkms_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.195.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/rangerkms_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.195.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.195.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.195.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/HTTP_127.8.195.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 16:54:54 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 16:54:54.582489 8974 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 16:54:54.610363 8974 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 16:54:55.539371 8974 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 16:54:55.539533 8974 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms
I20260812 16:54:55.539604 8974 mini_ranger_kms.cc:227] Using postgres at 127.8.195.148:60929
2026-08-12 16:54:55,925 [I] DB FLAVOR :POSTGRES
2026-08-12 16:54:55,925 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 16:54:55,925 [I] Checking connection
2026-08-12 16:54:55,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 16:54:56,173 [I] connection success
2026-08-12 16:54:56,173 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 16:54:56,173 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 16:54:56,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 16:54:56,414 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 16:54:56,414 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 16:54:56,414 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 16:54:56,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.195.148:60929/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 16:54:56,691 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 16:54:57.210927 8974 mini_ranger_kms.cc:310] Created kms service
I20260812 16:54:57.439951 8974 mini_ranger_kms.cc:326] Created kudu user
I20260812 16:54:57.488565 8974 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 16:54:57 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553694, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:54:57.947974 8974 mini_ranger_kms.cc:384] Added ranger policy
I20260812 16:54:57.948335 8974 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKFRmA3/build/asan/bin/postgresql.jar:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKFRmA3/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 16:54:57.948395 8974 mini_ranger_kms.cc:240] Using host: 127.8.195.148
I20260812 16:54:57.976989 8974 mini_ranger_kms.cc:276] Ranger KMS PID: 489
I20260812 16:54:57.977154 8974 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.195.148:40653
Aug 12, 2026 4:54:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 4:54:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 4:54:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 4:54:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 4:54:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 4:54:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 16:55:00 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786553700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 16:55:00 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:01.010633 8974 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.428s user 0.069s sys 0.356s
I20260812 16:55:01.010905 8974 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 16:55:01.011101 8974 mini_ranger_kms.cc:401] 127.8.195.148:40653/kms/v1/keys
Aug 12 16:55:01 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553700, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.195.148@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
Aug 12 16:55:03 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:55:03.718231 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41465
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:41465
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:04.071830 580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:04.072216 580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:04.072371 580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:04.079734 580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:04.079850 580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:04.079932 580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:04.080034 580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:04.080117 580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:04.090427 580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.195.148:40653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:41465
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:04.091763 580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:04.093124 580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:04.106387 594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:04.106341 601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:04.106359 596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:04.107553 580 server_base.cc:1034] running on GCE node
I20260812 16:55:04.108392 580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:04.110080 580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:04.111357 580 hybrid_clock.cc:648] HybridClock initialized: now 1786553704111280 us; error 43 us; skew 500 ppm
I20260812 16:55:04.111775 580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:04 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:04.118855 580 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:04.120544 580 webserver.cc:460] Webserver started at http://127.8.195.190:42317/ using document root <none> and password file <none>
I20260812 16:55:04.121346 580 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:04.121488 580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:04.121842 580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:55:04 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:04.172490 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "34cb532b46ea4456af67361570f5499f"
format_stamp: "Formatted at 2026-08-12 16:55:04 on dist-test-slave-z8x9"
server_key: "7e5273260dbe6a5933ea9d2c934a6f68"
server_key_iv: "411a39eea78635ad6bff06c2103268b8"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:04.173585 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "34cb532b46ea4456af67361570f5499f"
format_stamp: "Formatted at 2026-08-12 16:55:04 on dist-test-slave-z8x9"
server_key: "7e5273260dbe6a5933ea9d2c934a6f68"
server_key_iv: "411a39eea78635ad6bff06c2103268b8"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:04.179376 580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260812 16:55:04.196508 609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:04.198361 580 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 16:55:04.198562 580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "34cb532b46ea4456af67361570f5499f"
format_stamp: "Formatted at 2026-08-12 16:55:04 on dist-test-slave-z8x9"
server_key: "7e5273260dbe6a5933ea9d2c934a6f68"
server_key_iv: "411a39eea78635ad6bff06c2103268b8"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:04.198803 580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:04.223423 580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:04.227959 580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:04.228327 580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:04.247375 580 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:41465
I20260812 16:55:04.247406 695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:41465 every 8 connection(s)
I20260812 16:55:04.248956 580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:04.251325 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 580
I20260812 16:55:04.251549 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:55:04.252290 8974 external_mini_cluster.cc:1428] Setting key 5478590c2794407319c0b706b9604542
I20260812 16:55:04.253154 698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:04.266000 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Bootstrap starting.
Aug 12 16:55:04 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:04.269539 698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:04.270761 698 log.cc:824] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:04.274058 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: No bootstrap required, opened a new log
I20260812 16:55:04.275576 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:04.254445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.8.195.190:41465)
0812 16:55:04.255313 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:04.255325 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:04.255367 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:55:04.256934 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:04.256957 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:04.257465 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:04.257906 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:04.259354 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:04.260782 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:04.262226 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:04.262945 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:04.263174 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:04.267671 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:04.267680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:04.267695 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:04.267743 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:04.271149 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:04.271971 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:04.271978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:04.271988 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:04.272169 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:04.272647 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:04.272653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:04.272658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:04.273301 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:04.273685 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:04.274127 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 16:55:04.274548 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":402,"threads_started":1}
I20260812 16:55:04.282773 698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:04.283211 698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:04.283334 698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cb532b46ea4456af67361570f5499f, State: Initialized, Role: FOLLOWER
I20260812 16:55:04.284042 698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:04.284373 698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:04.284521 698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:04.284710 698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:04.286975 698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:04.287552 698 leader_election.cc:302] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34cb532b46ea4456af67361570f5499f; no voters:
I20260812 16:55:04.288332 698 leader_election.cc:288] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:55:04.288470 710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:04.289259 710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 LEADER]: Becoming Leader. State: Replica: 34cb532b46ea4456af67361570f5499f, State: Running, Role: LEADER
I20260812 16:55:04.289887 710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:04.290863 698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:04.292737 717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34cb532b46ea4456af67361570f5499f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } } }
I20260812 16:55:04.292757 711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34cb532b46ea4456af67361570f5499f. Latest consensus state: current_term: 1 leader_uuid: "34cb532b46ea4456af67361570f5499f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } } }
I20260812 16:55:04.293232 711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:04.293232 717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:04.294896 722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:04.301069 722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:04.302335 698 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 16:55:04.321559 722 catalog_manager.cc:1347] Generated new cluster ID: 6965a77cde214a1d986fe4968cbef690
I20260812 16:55:04.321709 722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:04.354544 722 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:55:04.357060 722 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:04.367861 722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Generated new TSK 0
I20260812 16:55:04.369123 722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:55:05.121572 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:05 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 16:55:05.577061 774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:05.577540 774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:05.577768 774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:05.585780 774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:05.585944 774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:05.586145 774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:05.597388 774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:05.599038 774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:05.600872 774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:05.620157 790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:05.620702 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:05.626822 774 server_base.cc:1034] running on GCE node
W20260812 16:55:05.627100 797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:05.628206 774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:05.630142 774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:05.631490 774 hybrid_clock.cc:648] HybridClock initialized: now 1786553705631376 us; error 106 us; skew 500 ppm
I20260812 16:55:05.632001 774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:05 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:05.636884 774 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:05.651443 774 webserver.cc:460] Webserver started at http://127.8.195.129:33755/ using document root <none> and password file <none>
I20260812 16:55:05.652554 774 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:05.652737 774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:05.653151 774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:55:05 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:05.682236 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "9e0900f259a34808abaef7b8e2be33ef"
format_stamp: "Formatted at 2026-08-12 16:55:05 on dist-test-slave-z8x9"
server_key: "28eddba1a0bb0008ff4b0aee5236fce9"
server_key_iv: "2f77c66a886eb1eb7416c6a961fc7b8e"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:05.683548 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "9e0900f259a34808abaef7b8e2be33ef"
format_stamp: "Formatted at 2026-08-12 16:55:05 on dist-test-slave-z8x9"
server_key: "28eddba1a0bb0008ff4b0aee5236fce9"
server_key_iv: "2f77c66a886eb1eb7416c6a961fc7b8e"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:05.704005 774 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.006s sys 0.009s
I20260812 16:55:05.736382 810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:05.738631 774 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.004s
I20260812 16:55:05.738873 774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "9e0900f259a34808abaef7b8e2be33ef"
format_stamp: "Formatted at 2026-08-12 16:55:05 on dist-test-slave-z8x9"
server_key: "28eddba1a0bb0008ff4b0aee5236fce9"
server_key_iv: "2f77c66a886eb1eb7416c6a961fc7b8e"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:05.739149 774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:05.787355 774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:05.793776 774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:05.794173 774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:05.795538 774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:05.797662 774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:05.797771 774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:05.797915 774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:05.797986 774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:05.864526 774 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:43889
I20260812 16:55:05.864573 1002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:43889 every 8 connection(s)
I20260812 16:55:05.866320 774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:05.867480 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 774
I20260812 16:55:05.867715 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:55:05.868571 8974 external_mini_cluster.cc:1428] Setting key 02c7f18b8a912a22d56120c4781cd6c3
Aug 12 16:55:05 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:05.918834 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:05.869845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:48473 (local address 127.8.195.190:41465)
0812 16:55:05.885643 (+ 15798us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:05.885650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:05.886936 (+ 1286us) server_negotiation.cc:407] Connection header received
0812 16:55:05.889482 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:05.889488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:05.889579 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:05.889677 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:05.892353 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:05.893679 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:05.896607 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:05.897410 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:05.897526 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:05.904949 (+ 7423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:05.904957 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:05.904962 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:05.905009 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:05.910264 (+ 5255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:05.913656 (+ 3392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:05.913665 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:05.913670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:05.913870 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:05.914539 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:05.914544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:05.914548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:05.914938 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:05.915081 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:05.918152 (+ 3071us) server_negotiation.cc:299] Negotiation successful
0812 16:55:05.918435 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15643,"thread_start_us":15503,"threads_started":1}
I20260812 16:55:05.919592 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:05.870309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.129:48473)
0812 16:55:05.886377 (+ 16068us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:05.886444 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:05.889066 (+ 2622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:05.890328 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:05.890342 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:05.891112 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:05.892113 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:05.892142 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:05.894123 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:05.894128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:05.896388 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:05.896405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:05.897971 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:05.897979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:05.898315 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:05.899240 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:05.899274 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:05.904637 (+ 5363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:05.911745 (+ 7108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:05.911756 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:05.911777 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:05.912509 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:05.914033 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:05.914039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:05.914042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:05.914363 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:05.915183 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:05.915192 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:05.915694 (+ 502us) client_negotiation.cc:769] Sending connection context
0812 16:55:05.916047 (+ 353us) client_negotiation.cc:240] Negotiation successful
0812 16:55:05.916671 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15680,"thread_start_us":15412,"threads_started":1}
I20260812 16:55:05.921604 1004 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:05.921998 1004 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:05.922966 1004 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:05.925931 637 ts_manager.cc:194] Registered new tserver with Master: 9e0900f259a34808abaef7b8e2be33ef (127.8.195.129:43889)
I20260812 16:55:05.928036 637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:48473
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Aug 12 16:55:05 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
W20260812 16:55:05.997458 8974 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.195.130: real 0.110s user 0.002s sys 0.024s
I20260812 16:55:05.998445 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:06 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
W20260812 16:55:06.529563 1013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:06.530053 1013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:06.530251 1013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:06.542375 1013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:06.542598 1013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:06.542843 1013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:06.559136 1013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:06.560877 1013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:06.562631 1013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:06.581585 1025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:06.583578 1027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:06.594195 1013 server_base.cc:1034] running on GCE node
W20260812 16:55:06.594996 1029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:06.595860 1013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:06.597229 1013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:06.598464 1013 hybrid_clock.cc:648] HybridClock initialized: now 1786553706598366 us; error 84 us; skew 500 ppm
I20260812 16:55:06.598843 1013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:06 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553706, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:06.602619 1013 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:06.606652 1013 webserver.cc:460] Webserver started at http://127.8.195.130:43455/ using document root <none> and password file <none>
I20260812 16:55:06.607440 1013 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:06.607600 1013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:06.607929 1013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:55:06 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553706, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:06.629128 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "803d134ee6d545e9afca524ed57f9f1d"
format_stamp: "Formatted at 2026-08-12 16:55:06 on dist-test-slave-z8x9"
server_key: "a75c3483bd39632a0c23ee8d8179d5f3"
server_key_iv: "edfc12b3f5e82c3860098461c4f134af"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:06.630125 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "803d134ee6d545e9afca524ed57f9f1d"
format_stamp: "Formatted at 2026-08-12 16:55:06 on dist-test-slave-z8x9"
server_key: "a75c3483bd39632a0c23ee8d8179d5f3"
server_key_iv: "edfc12b3f5e82c3860098461c4f134af"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:06.635689 1013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:55:06.651985 1048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:06.653679 1013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:06.653894 1013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "803d134ee6d545e9afca524ed57f9f1d"
format_stamp: "Formatted at 2026-08-12 16:55:06 on dist-test-slave-z8x9"
server_key: "a75c3483bd39632a0c23ee8d8179d5f3"
server_key_iv: "edfc12b3f5e82c3860098461c4f134af"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:06.654129 1013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:06.680922 1013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:06.685633 1013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:06.685983 1013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:06.687094 1013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:06.688627 1013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:06.688745 1013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:06.688905 1013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:06.688984 1013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:06.730628 1013 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:42489
I20260812 16:55:06.730741 1243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:42489 every 8 connection(s)
I20260812 16:55:06.732028 1013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:06.738673 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 1013
I20260812 16:55:06.738893 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:55:06.739403 8974 external_mini_cluster.cc:1428] Setting key 8d761ea9971349002609c4a7ab53ffd9
Aug 12 16:55:06 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553706, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:06.759335 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:06.735111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:42579 (local address 127.8.195.190:41465)
0812 16:55:06.735659 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:06.735666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:06.736912 (+ 1246us) server_negotiation.cc:407] Connection header received
0812 16:55:06.739584 (+ 2672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:06.739591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:06.739710 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:06.739803 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:06.742186 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:06.743374 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:06.745886 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:06.746614 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:06.746774 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:06.751785 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:06.751792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:06.751797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:06.751840 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:06.755616 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:06.756454 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:06.756465 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:06.756467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:06.756594 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:06.757116 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:06.757121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:06.757123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:06.757491 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:06.757662 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:06.758483 (+ 821us) server_negotiation.cc:299] Negotiation successful
0812 16:55:06.758723 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":267,"threads_started":1}
I20260812 16:55:06.760290 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:06.735725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.130:42579)
0812 16:55:06.736690 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:06.736756 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:06.739218 (+ 2462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:06.740392 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:06.740404 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:06.741053 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:06.741984 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:06.742010 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:06.743586 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:06.743591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:06.745715 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:06.745731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:06.747143 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:06.747152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:06.747455 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:06.748364 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:06.748404 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:06.751455 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:06.755801 (+ 4346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:06.755814 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:06.755829 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:06.756285 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:06.756713 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:06.756718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:06.756721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:06.756979 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:06.757628 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:06.757639 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:06.758070 (+ 431us) client_negotiation.cc:769] Sending connection context
0812 16:55:06.758380 (+ 310us) client_negotiation.cc:240] Negotiation successful
0812 16:55:06.758953 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":419,"threads_started":1}
I20260812 16:55:06.761922 1244 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:06.762255 1244 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:06.763007 1244 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:06.764591 637 ts_manager.cc:194] Registered new tserver with Master: 803d134ee6d545e9afca524ed57f9f1d (127.8.195.130:42489)
I20260812 16:55:06.765502 637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:42579
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:55:06.820123 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:06.932088 1004 heartbeater.cc:503] Master 127.8.195.190:41465 was elected leader, sending a full tablet report...
W20260812 16:55:07.191500 1254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:07.191859 1254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:07.192047 1254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:07.199743 1254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:07.199911 1254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:07.200099 1254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:07.210386 1254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:07.211884 1254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:07.213250 1254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:07.227372 1267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:07.227416 1268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:07.227444 1271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:07.228375 1254 server_base.cc:1034] running on GCE node
I20260812 16:55:07.229071 1254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:07.230288 1254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:07.231515 1254 hybrid_clock.cc:648] HybridClock initialized: now 1786553707231467 us; error 39 us; skew 500 ppm
I20260812 16:55:07.231899 1254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:07.235443 1254 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:07.237061 1254 webserver.cc:460] Webserver started at http://127.8.195.131:42227/ using document root <none> and password file <none>
I20260812 16:55:07.237882 1254 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:07.238013 1254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:07.238286 1254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:07.255118 1254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "06cbce4c29de4a31bae6bcfc1762a6f1"
format_stamp: "Formatted at 2026-08-12 16:55:07 on dist-test-slave-z8x9"
server_key: "5a7b12a147436b7e0040f28f48beccf6"
server_key_iv: "d15d889270977c864714cbec2a69bce1"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:07.256286 1254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "06cbce4c29de4a31bae6bcfc1762a6f1"
format_stamp: "Formatted at 2026-08-12 16:55:07 on dist-test-slave-z8x9"
server_key: "5a7b12a147436b7e0040f28f48beccf6"
server_key_iv: "d15d889270977c864714cbec2a69bce1"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:07.263091 1254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 16:55:07.278834 1284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:07.280813 1254 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260812 16:55:07.281059 1254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "06cbce4c29de4a31bae6bcfc1762a6f1"
format_stamp: "Formatted at 2026-08-12 16:55:07 on dist-test-slave-z8x9"
server_key: "5a7b12a147436b7e0040f28f48beccf6"
server_key_iv: "d15d889270977c864714cbec2a69bce1"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:07.281414 1254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:07.299825 1254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:07.304229 1254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:07.304553 1254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:07.305640 1254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:07.307152 1254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:07.307248 1254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:07.307386 1254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:07.307440 1254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:07.343118 1254 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:39265
I20260812 16:55:07.343160 1469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:39265 every 8 connection(s)
I20260812 16:55:07.344595 1254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:07.345221 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 1254
I20260812 16:55:07.345485 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:55:07.346184 8974 external_mini_cluster.cc:1428] Setting key 7051388b6d6941542a6ad8a56294e6dc
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:07.366918 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:07.347567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:55219 (local address 127.8.195.190:41465)
0812 16:55:07.348259 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:07.348266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:07.349403 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 16:55:07.351386 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:07.351392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:07.351489 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:07.351609 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:07.353126 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.353937 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.355556 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.356195 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.356366 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:07.360529 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:07.360535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:07.360538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:07.360573 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:07.363618 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.364402 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.364409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.364413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.364555 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.365038 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.365044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.365047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.365391 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:07.365566 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:07.366297 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 16:55:07.366543 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":376,"threads_started":1}
I20260812 16:55:07.367923 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:07.348389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.131:55219)
0812 16:55:07.349230 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:07.349292 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:07.351096 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:07.351883 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:07.351892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:07.352343 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:07.352948 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:07.352966 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.354100 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.354105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:07.355397 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:07.355410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.356523 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.356533 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:07.356892 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:07.357912 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:07.357948 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:07.360269 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:07.363773 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:07.363782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:07.363800 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:07.364272 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:07.364674 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:07.364679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:07.364683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:07.364916 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:07.365535 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:07.365544 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:07.365909 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 16:55:07.366195 (+ 286us) client_negotiation.cc:240] Negotiation successful
0812 16:55:07.366738 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":371,"threads_started":1}
I20260812 16:55:07.369437 1471 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:07.369815 1471 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:07.370599 1471 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:07.372087 637 ts_manager.cc:194] Registered new tserver with Master: 06cbce4c29de4a31bae6bcfc1762a6f1 (127.8.195.131:39265)
I20260812 16:55:07.373077 637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:55219
I20260812 16:55:07.375237 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:55:07.376930 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 774
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:55:07.395421 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:07.377971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.8.195.129:43889)
0812 16:55:07.378645 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:07.378651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:07.378676 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:55:07.378733 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:07.378738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:07.378922 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:07.379053 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:07.380574 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.381694 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.383333 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.383877 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.383978 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:07.389007 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:07.389021 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:07.389034 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:07.389074 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:07.392321 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.393070 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.393080 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.393084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.393330 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.393732 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.393738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.393741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.394101 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:07.394440 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:07.394759 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 16:55:07.395039 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":365,"threads_started":1}
I20260812 16:55:07.396834 993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:07.480703 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 774
I20260812 16:55:07.504815 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 1013
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:55:07.526995 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:07.510460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.8.195.130:42489)
0812 16:55:07.511038 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:07.511046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:07.511073 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:55:07.511136 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:07.511143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:07.511393 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:07.511529 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:07.512893 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.514072 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.515469 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.516069 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.516176 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:07.520924 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:07.520937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:07.520954 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:07.520995 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:07.524012 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.524777 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.524785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.524788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.524928 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.525437 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.525441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.525443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.525827 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:07.526078 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:07.526349 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 16:55:07.526565 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":324,"threads_started":1}
I20260812 16:55:07.528339 1237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:07.613517 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 1013
I20260812 16:55:07.640599 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 1254
Aug 12 16:55:07 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:55:07.657820 1484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:07.641776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53578 (local address 127.8.195.131:39265)
0812 16:55:07.642508 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:07.642521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:07.642550 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:55:07.642614 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:07.642621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:07.642858 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:07.642985 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:07.644348 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.645546 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.646792 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:07.647341 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:07.647443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:07.651951 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:07.651963 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:07.651978 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:07.652020 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:07.654945 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.655637 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.655644 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.655646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.655797 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:07.656126 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:07.656129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:07.656131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:07.656480 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:07.656755 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:07.657071 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 16:55:07.657375 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":402,"threads_started":1}
I20260812 16:55:07.659188 1461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:07.744319 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 1254
I20260812 16:55:07.771992 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 580
I20260812 16:55:07.773571 689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:07.883379 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 580
I20260812 16:55:07.921891 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41465
--webserver_interface=127.8.195.190
--webserver_port=42317
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:41465
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:08.263167 1488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:08.263509 1488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:08.263676 1488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:08.271005 1488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:08.271127 1488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:08.271261 1488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:08.271370 1488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:08.271471 1488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:08.282485 1488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:41465
--ranger_config_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:41465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=42317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:08.283878 1488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:08.285434 1488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:08.297863 1495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:08.298062 1494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:08.298471 1499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:08.299350 1488 server_base.cc:1034] running on GCE node
I20260812 16:55:08.300137 1488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:08.301599 1488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:08.302822 1488 hybrid_clock.cc:648] HybridClock initialized: now 1786553708302767 us; error 30 us; skew 500 ppm
I20260812 16:55:08.303294 1488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:08 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:08.306676 1488 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:08.308211 1488 webserver.cc:460] Webserver started at http://127.8.195.190:42317/ using document root <none> and password file <none>
I20260812 16:55:08.308907 1488 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:08.309065 1488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:55:08 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:08.356493 1488 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 16:55:08.359762 1519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:08.361306 1488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:08.361485 1488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "34cb532b46ea4456af67361570f5499f"
format_stamp: "Formatted at 2026-08-12 16:55:04 on dist-test-slave-z8x9"
server_key: "7e5273260dbe6a5933ea9d2c934a6f68"
server_key_iv: "411a39eea78635ad6bff06c2103268b8"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:08.362443 1488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:08.396474 1488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:08.400748 1488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:08.401077 1488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:08.420509 1488 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:41465
I20260812 16:55:08.420529 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:41465 every 8 connection(s)
I20260812 16:55:08.422163 1488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:08.429080 1619 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:55:08.431911 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 1488
I20260812 16:55:08.432463 1619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Bootstrap starting.
I20260812 16:55:08.432858 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:43889
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:41465
--webserver_port=33755
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:08.451656 1619 log.cc:824] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:08.460772 1619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:55:08.461392 1619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Bootstrap complete.
I20260812 16:55:08.470963 1619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:08.471881 1619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34cb532b46ea4456af67361570f5499f, State: Initialized, Role: FOLLOWER
I20260812 16:55:08.472568 1619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:08.472833 1619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:08.472960 1619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:08.473150 1619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:55:08.476958 1619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:08.477532 1619 leader_election.cc:302] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34cb532b46ea4456af67361570f5499f; no voters:
I20260812 16:55:08.478201 1619 leader_election.cc:288] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:55:08.478359 1628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:55:08.479497 1628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [term 2 LEADER]: Becoming Leader. State: Replica: 34cb532b46ea4456af67361570f5499f, State: Running, Role: LEADER
I20260812 16:55:08.480152 1628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } }
I20260812 16:55:08.480500 1619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:08.482843 1630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34cb532b46ea4456af67361570f5499f. Latest consensus state: current_term: 2 leader_uuid: "34cb532b46ea4456af67361570f5499f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } } }
I20260812 16:55:08.482969 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "34cb532b46ea4456af67361570f5499f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34cb532b46ea4456af67361570f5499f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 41465 } } }
I20260812 16:55:08.483472 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:08.483347 1630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:08.486744 1635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:08.493882 1635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:08.495437 1635 catalog_manager.cc:1259] Loaded cluster ID: 6965a77cde214a1d986fe4968cbef690
I20260812 16:55:08.495616 1635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:08.500919 1635 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:08.502889 1635 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Loaded TSK: 0
I20260812 16:55:08.504766 1635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:55:08.523166 1619 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260812 16:55:08.909219 1621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:08.909739 1621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:08.909960 1621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:08.923506 1621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:08.923760 1621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:08.924029 1621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:08.942334 1621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:43889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=33755
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:08.944485 1621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:08.946614 1621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:08.965564 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:08.970144 1684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:08.975885 1690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:08.977085 1621 server_base.cc:1034] running on GCE node
I20260812 16:55:08.978083 1621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:08.979604 1621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:08.981153 1621 hybrid_clock.cc:648] HybridClock initialized: now 1786553708981052 us; error 74 us; skew 500 ppm
I20260812 16:55:08.981626 1621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:08 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:08.986984 1621 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:09.001749 1621 webserver.cc:460] Webserver started at http://127.8.195.129:33755/ using document root <none> and password file <none>
I20260812 16:55:09.003002 1621 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:09.003301 1621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:09.054567 1621 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.000s
I20260812 16:55:09.066241 1707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:09.068454 1621 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.006s sys 0.006s
I20260812 16:55:09.068720 1621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "9e0900f259a34808abaef7b8e2be33ef"
format_stamp: "Formatted at 2026-08-12 16:55:05 on dist-test-slave-z8x9"
server_key: "28eddba1a0bb0008ff4b0aee5236fce9"
server_key_iv: "2f77c66a886eb1eb7416c6a961fc7b8e"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:09.070043 1621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:09.119823 1621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:09.134670 1621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:09.135139 1621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:09.137619 1621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:09.139529 1621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:09.139658 1621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:09.139797 1621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:09.139951 1621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 16:55:09.199302 1621 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:43889
I20260812 16:55:09.199404 1956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:43889 every 8 connection(s)
I20260812 16:55:09.201539 1621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:09.204035 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 1621
I20260812 16:55:09.204921 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:42489
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:41465
--webserver_port=43455
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:09.257545 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:09.209667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36793 (local address 127.8.195.190:41465)
0812 16:55:09.218098 (+ 8431us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:09.218124 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:09.226199 (+ 8075us) server_negotiation.cc:407] Connection header received
0812 16:55:09.230850 (+ 4651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:09.230878 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:09.231473 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:09.231996 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:09.234969 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.237013 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.239600 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.240647 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.240812 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:09.245658 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:09.245669 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:09.245690 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:09.245760 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:09.249913 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:09.252419 (+ 2506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:09.252430 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:09.252439 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:09.252656 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:09.253345 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:09.253352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:09.253356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:09.254116 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:09.254316 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:09.255446 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0812 16:55:09.255948 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8040,"spinlock_wait_cycles":40320,"thread_start_us":7936,"threads_started":1}
I20260812 16:55:09.260672 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:09.208313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.129:36793)
0812 16:55:09.225937 (+ 17624us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:09.226002 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:09.229982 (+ 3980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:09.232947 (+ 2965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:09.232959 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:09.233839 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:09.234714 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:09.234742 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.237314 (+ 2572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.237320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:09.239393 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:09.239425 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.241340 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.241349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:09.241627 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:09.242520 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:09.242558 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:09.245347 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:09.251519 (+ 6172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:09.251530 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:09.251547 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:09.252229 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:09.252855 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:09.252860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:09.252864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:09.253149 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:09.254419 (+ 1270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:09.254428 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:09.254870 (+ 442us) client_negotiation.cc:769] Sending connection context
0812 16:55:09.258780 (+ 3910us) client_negotiation.cc:240] Negotiation successful
0812 16:55:09.259376 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17226,"thread_start_us":17813,"threads_started":1}
I20260812 16:55:09.264624 1961 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:09.264971 1961 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:09.265931 1961 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
W20260812 16:55:09.269380 1961 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:41465 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:55:09.269619 1961 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:09.269907 1961 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:09.271271 1961 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:09.271615 1961 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
W20260812 16:55:09.272746 1961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:55:09.621803 1965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:09.622211 1965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:09.622392 1965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:09.630437 1965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:09.630600 1965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:09.630775 1965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:09.644018 1965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:42489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=43455
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:09.645560 1965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:09.646974 1965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:09.665027 1992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:09.665757 1993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:09.666242 1996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:09.667763 1965 server_base.cc:1034] running on GCE node
I20260812 16:55:09.668503 1965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:09.669652 1965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:09.670938 1965 hybrid_clock.cc:648] HybridClock initialized: now 1786553709670840 us; error 60 us; skew 500 ppm
I20260812 16:55:09.671326 1965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:09.674857 1965 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:09.676712 1965 webserver.cc:460] Webserver started at http://127.8.195.130:43455/ using document root <none> and password file <none>
I20260812 16:55:09.677616 1965 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:09.677790 1965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:09.702256 1965 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:55:09.712507 2009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:09.714524 1965 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.002s
I20260812 16:55:09.714756 1965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "803d134ee6d545e9afca524ed57f9f1d"
format_stamp: "Formatted at 2026-08-12 16:55:06 on dist-test-slave-z8x9"
server_key: "a75c3483bd39632a0c23ee8d8179d5f3"
server_key_iv: "edfc12b3f5e82c3860098461c4f134af"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:09.716305 1965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:09.732661 1965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:09.738446 1965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:09.738859 1965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:09.746702 1965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:09.752523 1965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:09.752748 1965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 16:55:09.752946 1965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:09.753074 1965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:09.840694 1965 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:42489
I20260812 16:55:09.840762 2220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:42489 every 8 connection(s)
I20260812 16:55:09.842850 1965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:09.847986 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 1965
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:09.848877 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.195.148:40653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:39265
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:41465
--webserver_port=42227
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:09 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:09.886024 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:09.850833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43633 (local address 127.8.195.190:41465)
0812 16:55:09.851352 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:09.851359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:09.863762 (+ 12403us) server_negotiation.cc:407] Connection header received
0812 16:55:09.865674 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:09.865681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:09.865780 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:09.865904 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:09.868245 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.869553 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.872178 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.872913 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.873043 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:09.878137 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:09.878144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:09.878149 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:09.878194 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:09.881972 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:09.883030 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:09.883036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:09.883040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:09.883202 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:09.883778 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:09.883784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:09.883787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:09.884168 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:09.884305 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:09.885390 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0812 16:55:09.885629 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":4619,"threads_started":1}
I20260812 16:55:09.888578 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:09.846955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.130:43633)
0812 16:55:09.861951 (+ 14996us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:09.862014 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:09.865140 (+ 3126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:09.866229 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:09.866245 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:09.867080 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:09.867977 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:09.868005 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.869777 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.869783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:09.871986 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:09.872012 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:09.873463 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:09.873471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:09.873775 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:09.874653 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:09.874695 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:09.877848 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:09.882155 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:09.882166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:09.882186 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:09.882854 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:09.883340 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:09.883345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:09.883349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:09.883625 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:09.884426 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:09.884436 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:09.884906 (+ 470us) client_negotiation.cc:769] Sending connection context
0812 16:55:09.886510 (+ 1604us) client_negotiation.cc:240] Negotiation successful
0812 16:55:09.887128 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14501,"thread_start_us":14227,"threads_started":1}
I20260812 16:55:09.890480 2221 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:09.890825 2221 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:09.891691 2221 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:09.894055 1556 ts_manager.cc:194] Registered new tserver with Master: 803d134ee6d545e9afca524ed57f9f1d (127.8.195.130:42489)
I20260812 16:55:09.896487 1556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:43633
I20260812 16:55:10.272953 1961 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:10.273631 1961 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:10.274981 1556 ts_manager.cc:194] Registered new tserver with Master: 9e0900f259a34808abaef7b8e2be33ef (127.8.195.129:43889)
I20260812 16:55:10.276151 1556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:36793
W20260812 16:55:10.301512 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:10.301947 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:10.302156 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:10.313956 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:10.314191 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:10.314410 2226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:10.330431 2226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.195.148:40653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:39265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=42227
--tserver_master_addrs=127.8.195.190:41465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:10.332214 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:10.334031 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 12 16:55:10 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
W20260812 16:55:10.358116 2233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:10.387845 2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:10.411129 2237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:10.412730 2226 server_base.cc:1034] running on GCE node
I20260812 16:55:10.413697 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:10.431684 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:10.433107 2226 hybrid_clock.cc:648] HybridClock initialized: now 1786553710433056 us; error 103 us; skew 500 ppm
I20260812 16:55:10.433584 2226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:10 dist-test-slave-z8x9 krb5kdc[32258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:10.438130 2226 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:10.445627 2226 webserver.cc:460] Webserver started at http://127.8.195.131:42227/ using document root <none> and password file <none>
I20260812 16:55:10.446568 2226 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:10.446753 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 16:55:10 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:10.476161 2226 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 16:55:10.482935 2254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:10.484970 2226 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260812 16:55:10.485252 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "06cbce4c29de4a31bae6bcfc1762a6f1"
format_stamp: "Formatted at 2026-08-12 16:55:07 on dist-test-slave-z8x9"
server_key: "5a7b12a147436b7e0040f28f48beccf6"
server_key_iv: "d15d889270977c864714cbec2a69bce1"
server_key_version: "kuduclusterkey@0"
I20260812 16:55:10.486828 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:10.524631 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:10.530388 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:10.530725 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:10.532100 2226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:10.534081 2226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:10.534211 2226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:10.534336 2226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:10.534416 2226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:10.603927 2226 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:39265
I20260812 16:55:10.603994 2464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:39265 every 8 connection(s)
I20260812 16:55:10.605557 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:10.611469 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 2226
Aug 12 16:55:10 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:10.661010 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.615965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.8.195.190:41465)
0812 16:55:10.637565 (+ 21600us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.637572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.637602 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:55:10.637679 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.637686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.637811 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.637976 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:10.640607 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.641617 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.647058 (+ 5441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.647752 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.647912 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.651656 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:10.651662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:10.651667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:10.651706 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:10.655557 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.656541 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.656547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.656551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.656719 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.658802 (+ 2083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.658810 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.658816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.659258 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:10.659560 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.660276 (+ 716us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.660545 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21414,"thread_start_us":7578,"threads_started":1}
I20260812 16:55:10.666270 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.612630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:52683 (local address 127.8.195.190:41465)
0812 16:55:10.623152 (+ 10522us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.623160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.639055 (+ 15895us) server_negotiation.cc:407] Connection header received
0812 16:55:10.644226 (+ 5171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.644233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.644335 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.644420 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:10.646582 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.647855 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.650480 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.651174 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.651316 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.657670 (+ 6354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:10.657677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:10.657682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:10.657722 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:10.661467 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.662526 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.662533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.662539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.662681 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.663325 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.663331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.663334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.663700 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:10.663841 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.665598 (+ 1757us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.665835 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10373,"thread_start_us":26697,"threads_started":1}
I20260812 16:55:10.667055 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.610452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:41465 (local address 127.8.195.131:52683)
0812 16:55:10.638852 (+ 28400us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:10.638933 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:10.643853 (+ 4920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:10.645094 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:10.645107 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:10.645654 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:10.646348 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.646366 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.648040 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.648045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.650271 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.650300 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.651669 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.651677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.652027 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.652835 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:10.652882 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:10.657368 (+ 4486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:10.661646 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:10.661658 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:10.661678 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:10.662371 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:10.662826 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:10.662830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:10.662833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:10.663163 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:10.663922 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:10.663932 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:10.664451 (+ 519us) client_negotiation.cc:769] Sending connection context
0812 16:55:10.664766 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 16:55:10.665425 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27990,"thread_start_us":27713,"threads_started":1}
I20260812 16:55:10.668661 2465 heartbeater.cc:347] Connected to a master server at 127.8.195.190:41465
I20260812 16:55:10.669004 2465 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:10.669958 2465 heartbeater.cc:511] Master 127.8.195.190:41465 requested a full tablet report, sending...
I20260812 16:55:10.671607 1556 ts_manager.cc:194] Registered new tserver with Master: 06cbce4c29de4a31bae6bcfc1762a6f1 (127.8.195.131:39265)
I20260812 16:55:10.672765 1556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:52683
I20260812 16:55:10.679003 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 16:55:10 dist-test-slave-z8x9 krb5kdc[32258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for HTTP/127.8.195.148@KRBTEST.COM
I20260812 16:55:10.702490 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.682701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.8.195.190:41465)
0812 16:55:10.682922 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.682932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.683483 (+ 551us) server_negotiation.cc:407] Connection header received
0812 16:55:10.683699 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.683704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.683800 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.683908 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:10.686730 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.687857 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.690669 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.691102 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.691193 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.695062 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:10.695069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:10.695074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:10.695114 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:10.698849 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.699822 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.699829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.699833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.700000 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:10.700505 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:10.700511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:10.700515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:10.700923 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:10.701186 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.701861 (+ 675us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.702129 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:55:10.710206 1556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:55:10.726624 1556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:55:10.849357 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.806481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45270)
0812 16:55:10.825586 (+ 19105us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:10.825676 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:10.825927 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:10.830447 (+ 4520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:10.830452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:10.830480 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:10.830936 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.830948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.839156 (+ 8208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.839161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.842036 (+ 2875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.842062 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.848316 (+ 6254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.848322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.848508 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.848597 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 16:55:10.848742 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 16:55:10.848941 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18722,"thread_start_us":35934,"threads_started":1}
I20260812 16:55:10.850665 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.808548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.8.195.131:39265)
0812 16:55:10.815693 (+ 7145us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.815704 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.826761 (+ 11057us) server_negotiation.cc:407] Connection header received
0812 16:55:10.828601 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.828615 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.828930 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.829129 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:10.831161 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.833414 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.845779 (+ 12365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.847419 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.847526 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.847789 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.850090 (+ 2301us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.850290 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6936,"thread_start_us":818,"threads_started":1}
I20260812 16:55:10.852494 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.807231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.8.195.129:43889)
0812 16:55:10.827264 (+ 20033us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.827274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.827303 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:55:10.827422 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.827430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.827723 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.827960 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:10.836978 (+ 9018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.838963 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.843157 (+ 4194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.844680 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.844811 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.845064 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.851925 (+ 6861us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.852134 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19867,"thread_start_us":19758,"threads_started":1}
I20260812 16:55:10.851747 2504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.807896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:39265 (local address 127.0.0.1:53586)
0812 16:55:10.826603 (+ 18707us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:10.826623 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:10.828310 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:10.829680 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:10.829690 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:10.829741 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:10.830230 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.830243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.833679 (+ 3436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.833688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.836667 (+ 2979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.836689 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.848028 (+ 11339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.848035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.849865 (+ 1830us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.849899 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:55:10.851138 (+ 1239us) client_negotiation.cc:240] Negotiation successful
0812 16:55:10.851292 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18537,"mutex_wait_us":1526,"thread_start_us":18899,"threads_started":1}
I20260812 16:55:10.854740 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.809511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.8.195.130:42489)
0812 16:55:10.822658 (+ 13147us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:10.822668 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:10.842938 (+ 20270us) server_negotiation.cc:407] Connection header received
0812 16:55:10.843136 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:10.843155 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:10.843473 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:10.843711 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:10.845715 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.847345 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.849886 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.853038 (+ 3152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.853196 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.853469 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:10.854195 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 16:55:10.854378 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12972,"thread_start_us":12681,"threads_started":1}
I20260812 16:55:10.855841 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:10.809199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:42489 (local address 127.0.0.1:60176)
0812 16:55:10.842792 (+ 33593us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:10.842809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:10.842973 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:10.845143 (+ 2170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:10.845148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:10.845195 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:10.845523 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.845531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.847529 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.847535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.849696 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:10.849715 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:10.853788 (+ 4073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:10.853795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:10.853965 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:10.853994 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 16:55:10.855288 (+ 1294us) client_negotiation.cc:240] Negotiation successful
0812 16:55:10.855449 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33457,"mutex_wait_us":3,"thread_start_us":15828,"threads_started":1}
I20260812 16:55:10.857780 2347 tablet_service.cc:1467] Processing CreateTablet for tablet b9646ff93d72487e8ec68c771ad20fcf (DEFAULT_TABLE table=test-table [id=fe50d587b9774103a0e70838db0133c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:10.859546 2347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9646ff93d72487e8ec68c771ad20fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:10.859951 2103 tablet_service.cc:1467] Processing CreateTablet for tablet b9646ff93d72487e8ec68c771ad20fcf (DEFAULT_TABLE table=test-table [id=fe50d587b9774103a0e70838db0133c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:10.861503 2103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9646ff93d72487e8ec68c771ad20fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:10.870803 1812 tablet_service.cc:1467] Processing CreateTablet for tablet b9646ff93d72487e8ec68c771ad20fcf (DEFAULT_TABLE table=test-table [id=fe50d587b9774103a0e70838db0133c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:10.874241 1812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9646ff93d72487e8ec68c771ad20fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:10.883387 2511 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Bootstrap starting.
I20260812 16:55:10.889747 2511 tablet_bootstrap.cc:654] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:10.891326 2511 log.cc:824] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:10.901708 2221 heartbeater.cc:503] Master 127.8.195.190:41465 was elected leader, sending a full tablet report...
I20260812 16:55:10.921514 2516 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Bootstrap starting.
I20260812 16:55:10.924230 2511 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: No bootstrap required, opened a new log
I20260812 16:55:10.924664 2511 ts_tablet_manager.cc:1397] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.028s
I20260812 16:55:10.927206 2516 tablet_bootstrap.cc:654] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:10.928686 2516 log.cc:824] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:10.934274 2517 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Bootstrap starting.
I20260812 16:55:10.939584 2516 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: No bootstrap required, opened a new log
I20260812 16:55:10.939978 2516 ts_tablet_manager.cc:1397] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20260812 16:55:10.940349 2511 raft_consensus.cc:348] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.941008 2511 raft_consensus.cc:374] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:10.941219 2511 raft_consensus.cc:729] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06cbce4c29de4a31bae6bcfc1762a6f1, State: Initialized, Role: FOLLOWER
I20260812 16:55:10.942072 2511 consensus_queue.cc:260] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.946543 2517 tablet_bootstrap.cc:654] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:10.947986 2517 log.cc:824] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:10.949380 2465 heartbeater.cc:503] Master 127.8.195.190:41465 was elected leader, sending a full tablet report...
I20260812 16:55:10.950443 2511 ts_tablet_manager.cc:1428] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Time spent starting tablet: real 0.025s user 0.010s sys 0.010s
I20260812 16:55:10.956852 2516 raft_consensus.cc:348] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.957319 2516 raft_consensus.cc:374] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:10.957494 2516 raft_consensus.cc:729] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803d134ee6d545e9afca524ed57f9f1d, State: Initialized, Role: FOLLOWER
I20260812 16:55:10.959756 2516 consensus_queue.cc:260] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.960610 2517 tablet_bootstrap.cc:492] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: No bootstrap required, opened a new log
I20260812 16:55:10.960996 2517 ts_tablet_manager.cc:1397] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.000s
I20260812 16:55:10.961807 2516 ts_tablet_manager.cc:1428] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Time spent starting tablet: real 0.022s user 0.008s sys 0.007s
I20260812 16:55:10.970284 2517 raft_consensus.cc:348] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.970772 2517 raft_consensus.cc:374] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:10.970892 2517 raft_consensus.cc:729] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0900f259a34808abaef7b8e2be33ef, State: Initialized, Role: FOLLOWER
I20260812 16:55:10.971489 2517 consensus_queue.cc:260] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:10.973153 2517 ts_tablet_manager.cc:1428] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Time spent starting tablet: real 0.012s user 0.005s sys 0.006s
I20260812 16:55:10.973326 1961 heartbeater.cc:503] Master 127.8.195.190:41465 was elected leader, sending a full tablet report...
W20260812 16:55:10.977051 1962 tablet.cc:2369] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:11.020917 2527 raft_consensus.cc:482] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:11.021292 2527 raft_consensus.cc:504] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:11.023257 2527 leader_election.cc:288] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 803d134ee6d545e9afca524ed57f9f1d (127.8.195.130:42489), 06cbce4c29de4a31bae6bcfc1762a6f1 (127.8.195.131:39265)
I20260812 16:55:11.031180 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.023885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:50935 (local address 127.8.195.131:39265)
0812 16:55:11.024319 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.024327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.024611 (+ 284us) server_negotiation.cc:407] Connection header received
0812 16:55:11.024865 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.024872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.024959 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.025081 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:11.025833 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.026994 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.029020 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.030105 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.030231 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.030283 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.030682 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.030775 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291}
I20260812 16:55:11.031214 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.023675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:39265 (local address 127.8.195.129:50935)
0812 16:55:11.024467 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.024499 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:11.024682 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:11.025192 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:11.025197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:11.025228 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:11.025671 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:11.025686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.027138 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.027161 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:11.028857 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:11.028869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.030323 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.030329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:11.030508 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.030545 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:55:11.030653 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 16:55:11.030810 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":360,"threads_started":1}
I20260812 16:55:11.031908 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.023791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:39035 (local address 127.8.195.130:42489)
0812 16:55:11.024151 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.024157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.024857 (+ 700us) server_negotiation.cc:407] Connection header received
0812 16:55:11.025256 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.025262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.025334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.025386 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:11.026406 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.027681 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.029730 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.030814 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.031001 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.031071 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.031561 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.031713 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260812 16:55:11.031917 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.023703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:42489 (local address 127.8.195.129:39035)
0812 16:55:11.024728 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.024747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:11.024961 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:11.025548 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:11.025551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:11.025608 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:11.026028 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:11.026041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.027850 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.027854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:11.029555 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:11.029569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.031178 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.031185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:11.031382 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.031416 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:55:11.031508 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:55:11.031592 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":666,"threads_started":1}
I20260812 16:55:11.032547 2392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9646ff93d72487e8ec68c771ad20fcf" candidate_uuid: "9e0900f259a34808abaef7b8e2be33ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" is_pre_election: true
I20260812 16:55:11.033043 2392 raft_consensus.cc:2454] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0900f259a34808abaef7b8e2be33ef in term 0.
I20260812 16:55:11.033752 1713 leader_election.cc:302] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cbce4c29de4a31bae6bcfc1762a6f1, 9e0900f259a34808abaef7b8e2be33ef; no voters:
I20260812 16:55:11.034162 2527 raft_consensus.cc:2781] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:55:11.034348 2527 raft_consensus.cc:482] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:55:11.034464 2527 raft_consensus.cc:3037] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:11.034401 2140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9646ff93d72487e8ec68c771ad20fcf" candidate_uuid: "9e0900f259a34808abaef7b8e2be33ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803d134ee6d545e9afca524ed57f9f1d" is_pre_election: true
I20260812 16:55:11.034829 2140 raft_consensus.cc:2454] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0900f259a34808abaef7b8e2be33ef in term 0.
I20260812 16:55:11.036949 2527 raft_consensus.cc:504] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:11.037602 2527 leader_election.cc:288] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [CANDIDATE]: Term 1 election: Requested vote from peers 803d134ee6d545e9afca524ed57f9f1d (127.8.195.130:42489), 06cbce4c29de4a31bae6bcfc1762a6f1 (127.8.195.131:39265)
I20260812 16:55:11.038141 2140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9646ff93d72487e8ec68c771ad20fcf" candidate_uuid: "9e0900f259a34808abaef7b8e2be33ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803d134ee6d545e9afca524ed57f9f1d"
I20260812 16:55:11.038141 2392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9646ff93d72487e8ec68c771ad20fcf" candidate_uuid: "9e0900f259a34808abaef7b8e2be33ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1"
I20260812 16:55:11.038359 2140 raft_consensus.cc:3037] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:11.038359 2392 raft_consensus.cc:3037] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:11.041735 2140 raft_consensus.cc:2454] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0900f259a34808abaef7b8e2be33ef in term 1.
I20260812 16:55:11.041729 2392 raft_consensus.cc:2454] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0900f259a34808abaef7b8e2be33ef in term 1.
I20260812 16:55:11.042187 1713 leader_election.cc:302] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cbce4c29de4a31bae6bcfc1762a6f1, 9e0900f259a34808abaef7b8e2be33ef; no voters:
I20260812 16:55:11.042503 2527 raft_consensus.cc:2781] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:11.042994 2527 raft_consensus.cc:686] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 1 LEADER]: Becoming Leader. State: Replica: 9e0900f259a34808abaef7b8e2be33ef, State: Running, Role: LEADER
I20260812 16:55:11.043485 2527 consensus_queue.cc:237] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } }
I20260812 16:55:11.048494 1556 catalog_manager.cc:5495] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0900f259a34808abaef7b8e2be33ef (127.8.195.129). New cstate: current_term: 1 leader_uuid: "9e0900f259a34808abaef7b8e2be33ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0900f259a34808abaef7b8e2be33ef" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 43889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260812 16:55:11.085307 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.079815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.8.195.129:43889)
0812 16:55:11.080029 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.080034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.080054 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:55:11.080249 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.080253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.080344 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.080535 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:55:11.080867 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.081775 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.083199 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.083575 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.083668 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.084144 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:55:11.084672 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:55:11.084803 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.084875 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.084938 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:55:11.093940 2392 raft_consensus.cc:1261] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 1 FOLLOWER]: Refusing update from remote peer 9e0900f259a34808abaef7b8e2be33ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:11.094003 2140 raft_consensus.cc:1261] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 1 FOLLOWER]: Refusing update from remote peer 9e0900f259a34808abaef7b8e2be33ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:11.094988 2551 consensus_queue.cc:1035] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "803d134ee6d545e9afca524ed57f9f1d" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:11.095371 2527 consensus_queue.cc:1035] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "06cbce4c29de4a31bae6bcfc1762a6f1" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:55:11.101349 2222 tablet.cc:2369] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:11.122720 1556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:11.122973 1556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:11.126762 1556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34cb532b46ea4456af67361570f5499f: Sending DeleteTablet for 3 replicas of tablet b9646ff93d72487e8ec68c771ad20fcf
I20260812 16:55:11.128043 1812 tablet_service.cc:1514] Processing DeleteTablet for tablet b9646ff93d72487e8ec68c771ad20fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:11 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:45270
I20260812 16:55:11.128084 2347 tablet_service.cc:1514] Processing DeleteTablet for tablet b9646ff93d72487e8ec68c771ad20fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:11 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53586
I20260812 16:55:11.129009 2574 tablet_replica.cc:330] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: stopping tablet replica
I20260812 16:55:11.129431 2103 tablet_service.cc:1514] Processing DeleteTablet for tablet b9646ff93d72487e8ec68c771ad20fcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:11 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:60176
I20260812 16:55:11.134099 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 1621
I20260812 16:55:11.135648 2574 raft_consensus.cc:2229] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:11.136226 2574 raft_consensus.cc:2258] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:11.139523 2574 ts_tablet_manager.cc:1905] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:11.147365 2577 tablet_replica.cc:330] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: stopping tablet replica
I20260812 16:55:11.148007 2577 raft_consensus.cc:2229] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:11.148670 2577 raft_consensus.cc:2258] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:11.154883 2577 ts_tablet_manager.cc:1905] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:11.161258 2574 ts_tablet_manager.cc:1918] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:11.161602 2574 log.cc:1196] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/b9646ff93d72487e8ec68c771ad20fcf
I20260812 16:55:11.162341 2574 ts_tablet_manager.cc:1939] T b9646ff93d72487e8ec68c771ad20fcf P 06cbce4c29de4a31bae6bcfc1762a6f1: Deleting consensus metadata
I20260812 16:55:11.163432 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.136254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.8.195.129:43889)
0812 16:55:11.136441 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.136447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.136470 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:55:11.136535 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.136543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.136650 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.136799 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:11.144436 (+ 7637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.145994 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.148572 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.149366 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.149514 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.153718 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:11.153736 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:11.153755 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:11.153817 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:11.158293 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.159361 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.159373 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.159380 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.159635 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.160159 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.160165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.160169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.160633 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:11.161002 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.162707 (+ 1705us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.162974 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:55:11.164719 1947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:11.165053 1530 catalog_manager.cc:4841] TS 06cbce4c29de4a31bae6bcfc1762a6f1 (127.8.195.131:39265): tablet b9646ff93d72487e8ec68c771ad20fcf (table test-table [id=fe50d587b9774103a0e70838db0133c6]) successfully deleted
I20260812 16:55:11.166401 2577 ts_tablet_manager.cc:1918] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:11.166603 2577 log.cc:1196] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/b9646ff93d72487e8ec68c771ad20fcf
I20260812 16:55:11.167268 2577 ts_tablet_manager.cc:1939] T b9646ff93d72487e8ec68c771ad20fcf P 803d134ee6d545e9afca524ed57f9f1d: Deleting consensus metadata
I20260812 16:55:11.169200 1526 catalog_manager.cc:4841] TS 803d134ee6d545e9afca524ed57f9f1d (127.8.195.130:42489): tablet b9646ff93d72487e8ec68c771ad20fcf (table test-table [id=fe50d587b9774103a0e70838db0133c6]) successfully deleted
I20260812 16:55:11.266651 2576 tablet_replica.cc:330] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef: stopping tablet replica
I20260812 16:55:11.267271 2576 raft_consensus.cc:2229] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:55:11.268025 2576 raft_consensus.cc:2258] T b9646ff93d72487e8ec68c771ad20fcf P 9e0900f259a34808abaef7b8e2be33ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:11.269874 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 1621
I20260812 16:55:11.298374 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 1965
W20260812 16:55:11.298388 1529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:43889 (error 108)
I20260812 16:55:11.300275 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.299860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45286)
0812 16:55:11.300025 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.300134 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:55:11.300511 1529 catalog_manager.cc:4584] TS 9e0900f259a34808abaef7b8e2be33ef (127.8.195.129:43889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9646ff93d72487e8ec68c771ad20fcf: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
I20260812 16:55:11.314816 2510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.299322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.8.195.130:42489)
0812 16:55:11.299625 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.299631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.299712 (+ 81us) server_negotiation.cc:407] Connection header received
0812 16:55:11.299949 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.299954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.300021 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.300142 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:11.302193 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.303018 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.304420 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.304921 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.305062 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.308029 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:11.308044 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:11.308060 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:11.308112 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:11.311400 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.312274 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.312283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.312285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.312440 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.312927 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.312934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.312937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.313435 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:11.313810 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.314139 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.314395 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260812 16:55:11.315609 2208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:11.354233 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.353713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45288)
0812 16:55:11.353940 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.354062 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 16:55:11.355366 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.355006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45292)
0812 16:55:11.355160 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.355258 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 16:55:11.410593 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 1965
I20260812 16:55:11.416085 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.415597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45298)
0812 16:55:11.415829 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.415926 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 16:55:11.417227 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.416814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45306)
0812 16:55:11.416970 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.417048 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 16:55:11.441483 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 2226
I20260812 16:55:11.456199 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.442302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.8.195.131:39265)
0812 16:55:11.442492 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:11.442497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:11.442518 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:55:11.442685 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:11.442689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:11.442755 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:11.442848 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:11.444203 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.444969 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.446458 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:11.446910 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:11.447018 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:11.450092 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:11.450111 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:11.450128 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:11.450165 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:11.453484 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.454215 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.454223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.454225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.454367 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:11.454709 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:11.454712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:11.454714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:11.455096 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:11.455507 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:11.455720 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 16:55:11.455897 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:55:11.456931 2460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:11.497733 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.497144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45308)
0812 16:55:11.497403 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.497526 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 16:55:11.498910 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.498461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45318)
0812 16:55:11.498699 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.498792 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 16:55:11.548836 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 2226
I20260812 16:55:11.578482 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 1488
I20260812 16:55:11.579761 1610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:11.696964 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:11.696036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:43889 (local address 127.0.0.1:45334)
0812 16:55:11.696441 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:11.696562 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
I20260812 16:55:11.697216 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 1488
I20260812 16:55:11.733055 8974 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 16:55:12.079869 8974 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 16:55:12.080149 8974 mini_ranger.cc:66] Stopping Ranger...
I20260812 16:55:12.426975 8974 mini_ranger.cc:68] Stopped Ranger
2026-08-12 16:55:12.427 UTC [32310] LOG: received smart shutdown request
2026-08-12 16:55:12.430 UTC [32310] LOG: background worker "logical replication launcher" (PID 32325) exited with exit code 1
2026-08-12 16:55:12.430 UTC [32318] LOG: shutting down
2026-08-12 16:55:12.614 UTC [32310] LOG: database system is shut down
2026-08-12T16:55:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56227 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:55:12 dist-test-slave-z8x9 krb5kdc[2605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:55:12 dist-test-slave-z8x9 krb5kdc[2605](info): set up 2 sockets
Aug 12 16:55:12 dist-test-slave-z8x9 krb5kdc[2605](info): commencing operation
krb5kdc: starting...
W20260812 16:55:14.184864 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:55:14 dist-test-slave-z8x9 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:55:14.497551 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45171
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:14.859021 2624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:14.859372 2624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:14.859468 2624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:14.867059 2624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:14.867192 2624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:14.867249 2624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:14.867309 2624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:14.867348 2624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:14.877741 2624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:14.879163 2624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:14.880475 2624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:14.893323 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:14.893347 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:14.893422 2636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:14.894315 2624 server_base.cc:1034] running on GCE node
I20260812 16:55:14.895110 2624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:14.896502 2624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:14.897703 2624 hybrid_clock.cc:648] HybridClock initialized: now 1786553714897630 us; error 59 us; skew 500 ppm
I20260812 16:55:14.898080 2624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:14 dist-test-slave-z8x9 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:14.906293 2624 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:14.907922 2624 webserver.cc:460] Webserver started at http://127.8.195.190:34599/ using document root <none> and password file <none>
I20260812 16:55:14.908658 2624 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:14.908805 2624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:14.909160 2624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:14.912283 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "2afcd6b0af4c41f8a46ec5c385ec627e"
format_stamp: "Formatted at 2026-08-12 16:55:14 on dist-test-slave-z8x9"
server_key: "8f9ebdcc35477fda0b69db9d429f73e7"
server_key_iv: "f07fb2a002afd53b79187901bed3d9af"
server_key_version: "encryptionkey@0"
I20260812 16:55:14.913126 2624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "2afcd6b0af4c41f8a46ec5c385ec627e"
format_stamp: "Formatted at 2026-08-12 16:55:14 on dist-test-slave-z8x9"
server_key: "8f9ebdcc35477fda0b69db9d429f73e7"
server_key_iv: "f07fb2a002afd53b79187901bed3d9af"
server_key_version: "encryptionkey@0"
I20260812 16:55:14.918462 2624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:55:14.922113 2648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:14.923717 2624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:55:14.923923 2624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "2afcd6b0af4c41f8a46ec5c385ec627e"
format_stamp: "Formatted at 2026-08-12 16:55:14 on dist-test-slave-z8x9"
server_key: "8f9ebdcc35477fda0b69db9d429f73e7"
server_key_iv: "f07fb2a002afd53b79187901bed3d9af"
server_key_version: "encryptionkey@0"
I20260812 16:55:14.924163 2624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:14.939119 2624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:14.943346 2624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:14.943670 2624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:14.963907 2624 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:45171
I20260812 16:55:14.963948 2738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:45171 every 8 connection(s)
I20260812 16:55:14.965453 2624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:14.968431 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 2624
I20260812 16:55:14.968674 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:55:14.969493 8974 external_mini_cluster.cc:1428] Setting key a5b497e61f6d55f02143f1b768b559cd
I20260812 16:55:14.969579 2740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:14.976517 2740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:55:14 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.195.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.189@KRBTEST.COM" created.
I20260812 16:55:15.001561 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:14.980042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.8.195.190:45171)
0812 16:55:14.981950 (+ 1908us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:14.981958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:14.982013 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:55:14.983620 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:14.983626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:14.983870 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:14.984074 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:14.985494 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:14.986750 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:14.988201 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:14.988780 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:14.988987 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:14.992676 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:14.992690 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:14.992709 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:14.992756 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:14.996979 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:14.997907 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:14.997914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:14.997916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:14.998075 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:14.998524 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:14.998528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:14.998530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:14.999080 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:14.999248 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:15.000029 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 16:55:15.000435 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":1653,"thread_start_us":1422,"threads_started":1}
I20260812 16:55:15.001626 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:14.980042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45171 (local address 127.0.0.1:35206)
0812 16:55:14.981060 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:14.981121 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:14.983224 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:14.984100 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:14.984107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:14.984867 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:14.985286 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:14.985312 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:14.986922 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:14.986927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:14.988059 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:14.988072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:14.989037 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:14.989044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:14.989257 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:14.990059 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:14.990080 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:14.992383 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:14.997220 (+ 4837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:14.997233 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:14.997243 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:14.997741 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:14.998186 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:14.998189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:14.998191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:14.998416 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:14.999222 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:14.999229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:14.999463 (+ 234us) client_negotiation.cc:769] Sending connection context
0812 16:55:14.999781 (+ 318us) client_negotiation.cc:240] Negotiation successful
0812 16:55:15.000296 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":483,"threads_started":1}
I20260812 16:55:15.004830 2740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:55:15.006225 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.005682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56664)
0812 16:55:15.005897 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.006052 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 16:55:15.006583 2654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
I20260812 16:55:15.008404 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.007930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56674)
0812 16:55:15.008103 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.008201 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:55:15.008621 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:43181: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
I20260812 16:55:15.036993 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:43181
--webserver_interface=127.8.195.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:15.058061 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } attempt: 1
I20260812 16:55:15.059569 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.058942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56680)
0812 16:55:15.059185 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.059381 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 16:55:15.060602 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.060234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56682)
0812 16:55:15.060384 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.060466 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:55:15.060788 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:43181: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
I20260812 16:55:15.129184 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } attempt: 2
I20260812 16:55:15.130424 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.129903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56688)
0812 16:55:15.130170 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.130265 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 16:55:15.131531 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.131163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56700)
0812 16:55:15.131300 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.131391 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 16:55:15.131769 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:43181: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
I20260812 16:55:15.223088 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } attempt: 3
I20260812 16:55:15.225445 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.224762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56712)
0812 16:55:15.225088 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.225228 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260812 16:55:15.234264 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.233559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56718)
0812 16:55:15.233905 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.234035 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260812 16:55:15.234568 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:43181: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
I20260812 16:55:15.377933 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } attempt: 4
I20260812 16:55:15.379292 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.378665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56724)
0812 16:55:15.379047 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.379140 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260812 16:55:15.380481 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.380071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56736)
0812 16:55:15.380260 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.380350 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:55:15.380671 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:43181: Network error: Client connection negotiation failed: client connection to 127.8.195.189:43181: connect: Connection refused (error 111)
W20260812 16:55:15.422366 2756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:15.422735 2756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:15.422902 2756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:15.430891 2756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:15.431047 2756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:15.431154 2756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:15.431252 2756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:15.431341 2756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:15.441829 2756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.195.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:15.443255 2756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:15.444692 2756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:15.457304 2770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:15.457324 2771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:15.457304 2774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:15.457967 2756 server_base.cc:1034] running on GCE node
I20260812 16:55:15.458837 2756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:15.460388 2756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:15.461635 2756 hybrid_clock.cc:648] HybridClock initialized: now 1786553715461564 us; error 63 us; skew 500 ppm
I20260812 16:55:15.462039 2756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:15 dist-test-slave-z8x9 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553715, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:15.465523 2756 init.cc:377] Logged in from keytab as kudu/127.8.195.189@KRBTEST.COM (short username kudu)
I20260812 16:55:15.467067 2756 webserver.cc:460] Webserver started at http://127.8.195.189:39019/ using document root <none> and password file <none>
I20260812 16:55:15.467779 2756 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:15.467887 2756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:15.468138 2756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:15.471195 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data/instance:
uuid: "93d2772e869c416d89699fa4343c2611"
format_stamp: "Formatted at 2026-08-12 16:55:15 on dist-test-slave-z8x9"
server_key: "9cbb48e8c1395a1d18634b22af007514"
server_key_iv: "59ffd572f05b3266862c55a69ebb2b24"
server_key_version: "encryptionkey@0"
I20260812 16:55:15.471969 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal/instance:
uuid: "93d2772e869c416d89699fa4343c2611"
format_stamp: "Formatted at 2026-08-12 16:55:15 on dist-test-slave-z8x9"
server_key: "9cbb48e8c1395a1d18634b22af007514"
server_key_iv: "59ffd572f05b3266862c55a69ebb2b24"
server_key_version: "encryptionkey@0"
I20260812 16:55:15.477360 2756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260812 16:55:15.480842 2788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:15.482332 2756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:55:15.482542 2756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "93d2772e869c416d89699fa4343c2611"
format_stamp: "Formatted at 2026-08-12 16:55:15 on dist-test-slave-z8x9"
server_key: "9cbb48e8c1395a1d18634b22af007514"
server_key_iv: "59ffd572f05b3266862c55a69ebb2b24"
server_key_version: "encryptionkey@0"
I20260812 16:55:15.482757 2756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:15.507484 2756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:15.512265 2756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:15.512591 2756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:15.532065 2756 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.189:43181
I20260812 16:55:15.532104 2885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.189:43181 every 8 connection(s)
I20260812 16:55:15.533588 2756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
I20260812 16:55:15.537747 2888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:15.541028 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 2756
I20260812 16:55:15.541312 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-1/wal/instance
I20260812 16:55:15.541883 8974 external_mini_cluster.cc:1428] Setting key b69162c2eb13703732496108852a5f3e
I20260812 16:55:15.547314 2888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:55:15 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553715, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.195.188@KRBTEST.COM; defaulting to no policy
I20260812 16:55:15.572098 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.551022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.8.195.190:45171)
0812 16:55:15.551618 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:15.551635 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:15.552702 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 16:55:15.554715 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:15.554722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:15.554821 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:15.554966 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:15.556915 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.557812 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.560404 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.560825 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.560929 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.565479 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:15.565485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:15.565488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:15.565539 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:15.568564 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.569535 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.569543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.569551 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.569683 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.570165 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.570170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.570173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.570482 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:15.570652 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:15.571465 (+ 813us) server_negotiation.cc:299] Negotiation successful
0812 16:55:15.571703 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":267,"threads_started":1}
I20260812 16:55:15.573140 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.551660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45171 (local address 127.0.0.1:35222)
0812 16:55:15.552511 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.552561 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:15.554419 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:15.555163 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:15.555171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:15.555871 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:15.556673 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.556701 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.557987 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.557995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.560235 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.560253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.561064 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.561074 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.561433 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.562235 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:15.562269 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:15.565129 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:15.568748 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.568758 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.568779 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.569395 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.569818 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.569823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.569832 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.570025 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.570627 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:15.570637 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:15.571051 (+ 414us) client_negotiation.cc:769] Sending connection context
0812 16:55:15.571342 (+ 291us) client_negotiation.cc:240] Negotiation successful
0812 16:55:15.571924 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":374,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.188@KRBTEST.COM" created.
I20260812 16:55:15.575213 2888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:55:15 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553715, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:55:15.592839 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.576321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.8.195.189:43181)
0812 16:55:15.577095 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:15.577101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:15.577140 (+ 39us) server_negotiation.cc:407] Connection header received
0812 16:55:15.577286 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:15.577310 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:15.577531 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:15.577676 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:15.578689 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.579742 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.581605 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.582139 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.582237 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.586197 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:15.586216 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:15.586237 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:15.586281 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:15.589581 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.590458 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.590465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.590467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.590584 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.591084 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.591087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.591090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.591458 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:15.591628 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:15.592125 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 16:55:15.592495 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":617,"thread_start_us":478,"threads_started":1}
I20260812 16:55:15.592883 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.576054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56740)
0812 16:55:15.576260 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.576283 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:15.576405 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:15.577704 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:15.577712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:15.578271 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:15.578499 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.578514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.579914 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.579922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.581465 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.581485 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.582309 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.582320 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.582573 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.583223 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:15.583252 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:15.585918 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:15.589726 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.589736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.589743 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.590324 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.590695 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.590703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.590709 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.590960 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.591597 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:15.591605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:15.591791 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 16:55:15.591957 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 16:55:15.592333 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 16:55:15.595446 2888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:55:15.596654 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.596187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33918)
0812 16:55:15.596354 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.596489 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:55:15.596993 2796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.598699 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.598374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33920)
0812 16:55:15.598514 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.598596 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:55:15.598942 2888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
I20260812 16:55:15.616253 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:34607
--webserver_interface=127.8.195.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:15.648317 2888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 1
I20260812 16:55:15.649590 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.649057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33928)
0812 16:55:15.649308 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.649433 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 16:55:15.650696 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.650310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33940)
0812 16:55:15.650485 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.650567 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 16:55:15.650997 2888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.679953 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } attempt: 5
Aug 12 16:55:15 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:55:15.695571 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.681027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.8.195.189:43181)
0812 16:55:15.681294 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:15.681299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:15.681317 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:55:15.681374 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:15.681378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:15.681438 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:15.681523 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:15.682575 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.683299 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.685093 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.685698 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.685824 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.689833 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:15.689839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:15.689843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:15.689880 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:15.692666 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.693488 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.693494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.693499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.693653 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:15.694077 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:15.694081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:15.694083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:15.694381 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:15.694552 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:15.694970 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 16:55:15.695211 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:55:15.695580 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.680719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56754)
0812 16:55:15.680996 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.681025 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:15.681199 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:15.681600 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:15.681609 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:15.682148 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:15.682398 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.682412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.683449 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.683455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.684875 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:15.684895 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:15.685881 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:15.685887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:15.686086 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:15.686859 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:15.686884 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:15.689594 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:15.692830 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.692836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.692841 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.693355 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.693780 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:15.693785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:15.693789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:15.693943 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:15.694511 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:15.694516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:15.694694 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 16:55:15.694840 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 16:55:15.695212 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 16:55:15.696770 2740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:55:15.698324 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.697768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33950)
0812 16:55:15.697986 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.698101 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 16:55:15.699448 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.699016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33962)
0812 16:55:15.699193 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.699294 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 16:55:15.699694 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.719266 2888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 2
I20260812 16:55:15.720517 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.720033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33976)
0812 16:55:15.720244 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.720355 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:55:15.721572 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.721227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33984)
0812 16:55:15.721380 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.721457 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:55:15.721769 2888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.750952 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 1
I20260812 16:55:15.752245 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.751774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:33996)
0812 16:55:15.751979 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.752073 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 16:55:15.753320 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.752919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34002)
0812 16:55:15.753068 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.753150 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:55:15.753532 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.813045 2888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 3
I20260812 16:55:15.814463 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.813850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34018)
0812 16:55:15.814130 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.814251 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 16:55:15.816047 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.815412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34020)
0812 16:55:15.815783 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.815890 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
W20260812 16:55:15.816430 2888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.821748 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 2
I20260812 16:55:15.822970 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.822443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34036)
0812 16:55:15.822672 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.822788 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 16:55:15.824066 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.823683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34040)
0812 16:55:15.823845 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.823936 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 16:55:15.824282 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.930500 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 3
I20260812 16:55:15.931736 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.931212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34042)
0812 16:55:15.931461 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.931559 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 16:55:15.932957 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.932539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34056)
0812 16:55:15.932764 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.932849 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260812 16:55:15.933207 2740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
I20260812 16:55:15.959709 2888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 4
I20260812 16:55:15.961076 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.960478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34062)
0812 16:55:15.960763 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.960872 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 16:55:15.962224 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:15.961817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34070)
0812 16:55:15.961975 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:15.962067 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 16:55:15.962474 2888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:34607: Network error: Client connection negotiation failed: client connection to 127.8.195.188:34607: connect: Connection refused (error 111)
W20260812 16:55:15.983954 2899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:15.984275 2899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:15.984458 2899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:15.992914 2899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:15.993060 2899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:15.993146 2899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:15.993269 2899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:15.993357 2899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:16.003841 2899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45171,127.8.195.189:43181,127.8.195.188:34607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:34607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.195.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:16.005429 2899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:16.006825 2899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:16.019342 2910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:16.019312 2911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:16.019407 2914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:16.020435 2899 server_base.cc:1034] running on GCE node
I20260812 16:55:16.021363 2899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:16.022878 2899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:16.024237 2899 hybrid_clock.cc:648] HybridClock initialized: now 1786553716024176 us; error 52 us; skew 500 ppm
I20260812 16:55:16.024637 2899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:16.028499 2899 init.cc:377] Logged in from keytab as kudu/127.8.195.188@KRBTEST.COM (short username kudu)
I20260812 16:55:16.030164 2899 webserver.cc:460] Webserver started at http://127.8.195.188:39585/ using document root <none> and password file <none>
I20260812 16:55:16.030926 2899 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:16.031039 2899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:16.031373 2899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:16.034623 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data/instance:
uuid: "d0ba09d17f26479d833e02f5734ba7e2"
format_stamp: "Formatted at 2026-08-12 16:55:16 on dist-test-slave-z8x9"
server_key: "315077e49c343d462db994c5426ff9a1"
server_key_iv: "1a721ed60bfa1306ebaa2489cb7e7638"
server_key_version: "encryptionkey@0"
I20260812 16:55:16.035503 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal/instance:
uuid: "d0ba09d17f26479d833e02f5734ba7e2"
format_stamp: "Formatted at 2026-08-12 16:55:16 on dist-test-slave-z8x9"
server_key: "315077e49c343d462db994c5426ff9a1"
server_key_iv: "1a721ed60bfa1306ebaa2489cb7e7638"
server_key_version: "encryptionkey@0"
I20260812 16:55:16.041244 2899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 16:55:16.044857 2933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:16.046543 2899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:55:16.046739 2899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "d0ba09d17f26479d833e02f5734ba7e2"
format_stamp: "Formatted at 2026-08-12 16:55:16 on dist-test-slave-z8x9"
server_key: "315077e49c343d462db994c5426ff9a1"
server_key_iv: "1a721ed60bfa1306ebaa2489cb7e7638"
server_key_version: "encryptionkey@0"
I20260812 16:55:16.046977 2899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:16.083532 2899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:16.089143 2899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:16.089535 2899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:16.109345 2899 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.188:34607
I20260812 16:55:16.109400 3020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.188:34607 every 8 connection(s)
I20260812 16:55:16.110448 2740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 4
I20260812 16:55:16.111052 2899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
I20260812 16:55:16.115322 3021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:16.118045 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 2899
I20260812 16:55:16.118284 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786553491558846-8974-0/minicluster-data/master-2/wal/instance
I20260812 16:55:16.118902 8974 external_mini_cluster.cc:1428] Setting key 1b7a5dceb61e176c0793beef6845d38b
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:55:16.125442 3021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:55:16.133481 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.111201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34086)
0812 16:55:16.111380 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:16.111403 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:16.111546 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:16.115960 (+ 4414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:16.115966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:16.116508 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:16.116733 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.116748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.118614 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.118622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.119664 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.119685 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.122033 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.122040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.122272 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.122961 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:16.122980 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:16.125475 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:16.130525 (+ 5050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.130530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.130534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.130955 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.131431 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.131436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.131440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.131584 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.132349 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:16.132354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:16.132533 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 16:55:16.132704 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 16:55:16.133046 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 16:55:16.134523 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.112489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.8.195.188:34607)
0812 16:55:16.113359 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.113372 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.113420 (+ 48us) server_negotiation.cc:407] Connection header received
0812 16:55:16.115090 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.115113 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.115548 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.115913 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.116932 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.118431 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.120451 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.121842 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.122074 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.125859 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.125872 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.125894 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.125955 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.130403 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.131093 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.131102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.131110 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.131301 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.131697 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.131702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.131706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.132203 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.132401 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.132963 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.133451 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":393,"threads_started":1}
I20260812 16:55:16.141162 2740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: Bootstrap starting.
I20260812 16:55:16.143743 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.119967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34090 (local address 127.8.195.188:34607)
0812 16:55:16.120636 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.120648 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.121847 (+ 1199us) server_negotiation.cc:407] Connection header received
0812 16:55:16.122099 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.122107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.122188 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.122267 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.124423 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.125772 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.127368 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.128081 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.128215 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.135143 (+ 6928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.135155 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.135162 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.135215 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.139908 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.140877 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.140886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.140892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.141108 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.141628 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.141636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.141642 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.142127 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.142465 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.142803 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.143069 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":326,"threads_started":1}
I20260812 16:55:16.145505 2740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:16.147140 2740 log.cc:824] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:16.150457 2740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: No bootstrap required, opened a new log
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:16.162266 2740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.162670 2740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:16.162799 2740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2afcd6b0af4c41f8a46ec5c385ec627e, State: Initialized, Role: FOLLOWER
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:16.163632 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.129142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45171 (local address 127.0.0.1:35232)
0812 16:55:16.130102 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:16.130146 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:16.130320 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:16.143285 (+ 12965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:16.143292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:16.144270 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:16.144577 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.144587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.146486 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.146495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.148750 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.148778 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.150312 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.150319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.150548 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.151498 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:16.151524 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:16.154642 (+ 3118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:16.159680 (+ 5038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.159702 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.159712 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.160781 (+ 1069us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.161317 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.161325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.161329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.161546 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.162320 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:16.162331 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:16.162558 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 16:55:16.162805 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 16:55:16.163246 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"spinlock_wait_cycles":20352,"thread_start_us":614,"threads_started":1}
I20260812 16:55:16.163548 2740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.165114 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.129663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.8.195.190:45171)
0812 16:55:16.142773 (+ 13110us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.142779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.142797 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:55:16.142857 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.142862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.142940 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.144719 (+ 1779us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.144928 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.146200 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.149040 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.150049 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.150294 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.154902 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.154910 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.154915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.154955 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.159474 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.160946 (+ 1472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.160958 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.160962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.161139 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.161688 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.161693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.161696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.162133 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.162304 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.164264 (+ 1960us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.164559 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12939,"spinlock_wait_cycles":30592,"thread_start_us":13389,"threads_started":1}
I20260812 16:55:16.164721 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.165890 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:55:16.166429 2740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:16.166944 3021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:55:16.175202 3038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.145867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.8.195.190:45171)
0812 16:55:16.153784 (+ 7917us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.153791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.153817 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:55:16.153881 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.153886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.153975 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.154101 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.155766 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.156891 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.158598 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.159120 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.159288 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.165051 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.165061 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.165071 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.165122 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.170165 (+ 5043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.171416 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.171422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.171426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.171694 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.172310 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.172317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.172320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.172785 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.174362 (+ 1577us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.174497 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.174774 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7744,"thread_start_us":7495,"threads_started":1}
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:55:16.179062 2624 master_runner.cc:186] Error getting master registration for 127.8.195.189:43181: OK, instance_id { permanent_uuid: "93d2772e869c416d89699fa4343c2611" instance_seqno: 1786553715526670 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:55:16.186417 2624 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:55:16.188685 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.167803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.8.195.189:43181)
0812 16:55:16.168013 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.168019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.168298 (+ 279us) server_negotiation.cc:407] Connection header received
0812 16:55:16.168418 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.168426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.168526 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.168633 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.169971 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.170732 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.172592 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.173268 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.173381 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.179777 (+ 6396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.179787 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.179795 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.179849 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.184865 (+ 5016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.185952 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.185959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.185963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.186162 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.186683 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.186691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.186695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.187162 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.187349 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.187931 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.188204 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 16:55:16.189801 3062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:55:16.189975 3062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:55:16.190006 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.167819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:43181 (local address 127.0.0.1:56768)
0812 16:55:16.168023 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:16.168044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:16.168175 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:16.168849 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:16.168856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:16.169483 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:16.169737 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.169752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.170951 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.170958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.172374 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.172387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.173652 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.173658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.173865 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.174601 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:16.174628 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:16.179474 (+ 4846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:16.185059 (+ 5585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.185065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.185071 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.185721 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.186349 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.186354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.186357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.186559 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.187421 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:16.187428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:16.187615 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 16:55:16.189203 (+ 1588us) client_negotiation.cc:240] Negotiation successful
0812 16:55:16.189543 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 16:55:16.191325 3021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
W20260812 16:55:16.196749 2656 tablet.cc:2369] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553716, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:55:16.205586 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.178407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.8.195.189:43181)
0812 16:55:16.185767 (+ 7360us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.185776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.185807 (+ 31us) server_negotiation.cc:407] Connection header received
0812 16:55:16.185876 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.185885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.186051 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.188278 (+ 2227us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.188598 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.189905 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.191620 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.192262 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.192394 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.198224 (+ 5830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.198230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.198234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.198272 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.201937 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.202764 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.202771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.202775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.202944 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.203450 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.203458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.203462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.203857 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.204126 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.204823 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.205094 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7179,"thread_start_us":6940,"threads_started":1}
I20260812 16:55:16.209074 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.192154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34092)
0812 16:55:16.192353 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:16.192374 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:16.192480 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:16.192910 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:16.192915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:16.193473 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:16.193680 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.193694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.195123 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.195129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.196608 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.196624 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.197850 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.197856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.198043 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.198778 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:16.198804 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:16.201714 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:16.206007 (+ 4293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.206013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.206017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.206548 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.207008 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.207015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.207019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.207199 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.207881 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:16.207887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:16.208050 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 16:55:16.208252 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 16:55:16.208622 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 16:55:16.209314 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.192404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.8.195.188:34607)
0812 16:55:16.192594 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.192600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.192618 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:55:16.192682 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.192687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.192758 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.192865 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.193858 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.194972 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.196764 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.197504 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.197614 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.201963 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.201970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.201975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.202021 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.205589 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.206677 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.206685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.206688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.206866 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.207327 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.207332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.207335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.207741 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.207897 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.208385 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.208831 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":58}
I20260812 16:55:16.211539 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:55:16.214854 3021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: Bootstrap starting.
I20260812 16:55:16.218398 3021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:16.219902 3021 log.cc:824] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:16.223717 3021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: No bootstrap required, opened a new log
I20260812 16:55:16.237205 3021 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.237843 3021 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:16.238049 3021 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ba09d17f26479d833e02f5734ba7e2, State: Initialized, Role: FOLLOWER
I20260812 16:55:16.238894 3021 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.250279 3038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.221627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.8.195.190:45171)
0812 16:55:16.221914 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.221924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.227771 (+ 5847us) server_negotiation.cc:407] Connection header received
0812 16:55:16.228074 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.228082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.228194 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.228271 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.231003 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.232202 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.234453 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.235168 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.235278 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.240015 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.240025 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.240033 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.240098 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.245343 (+ 5245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.246749 (+ 1406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.246759 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.246765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.246981 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.247609 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.247618 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.247624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.248120 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.248471 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.249393 (+ 922us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.249747 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 16:55:16.261711 2888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } attempt: 5
I20260812 16:55:16.267762 3021 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:16.265645 3078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.269976 3078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 16:55:16 dist-test-slave-z8x9 krb5kdc[2605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553715, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:55:16.274791 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.237067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.8.195.188:34607)
0812 16:55:16.237304 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.237309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.252015 (+ 14706us) server_negotiation.cc:407] Connection header received
0812 16:55:16.252346 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.252356 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.252473 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.252567 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.255302 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.256815 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.259406 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.260426 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.260570 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.264883 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.264894 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.264902 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.264959 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.269104 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.271263 (+ 2159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.271273 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.271279 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.271504 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.272153 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.272162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.272168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.272718 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.273039 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.273832 (+ 793us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.274182 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260812 16:55:16.279220 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.222508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.8.195.189:43181)
0812 16:55:16.222720 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.222726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.236882 (+ 14156us) server_negotiation.cc:407] Connection header received
0812 16:55:16.237257 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.237263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.237357 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.237440 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.246515 (+ 9075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.248014 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.250992 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.251911 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.252054 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.256710 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.256717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.256724 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.256769 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.260529 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.261550 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.261557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.261561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.261736 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.262273 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.262278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.262282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.262654 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.262929 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.278588 (+ 15659us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.278869 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:55:16.282030 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.262348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:34607 (local address 127.0.0.1:34116)
0812 16:55:16.262755 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:16.262776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:16.262896 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:16.263820 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:16.263826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:16.264313 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:16.264522 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.264536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.266094 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.266100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.267581 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:16.267598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.268982 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.268989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:16.269238 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.269886 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:16.269911 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:16.272739 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:16.277060 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.277067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.277071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.277720 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.278171 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:16.278176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:16.278179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:16.278350 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:16.280869 (+ 2519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:16.280876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:16.281084 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 16:55:16.281271 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 16:55:16.281622 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260812 16:55:16.282696 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:16.263144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.8.195.188:34607)
0812 16:55:16.263335 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:16.263343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:16.263365 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:55:16.263423 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:16.263428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:16.263506 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:16.263603 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:16.264727 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.265878 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.267797 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:16.268545 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:16.268746 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:16.272989 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:16.272998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:16.273003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:16.273041 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:16.276868 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.277880 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.277886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.277890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.278033 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:16.280090 (+ 2057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:16.280099 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:16.280105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:16.280659 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:16.281717 (+ 1058us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:16.281847 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 16:55:16.282084 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:55:16.289866 2888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: Bootstrap starting.
I20260812 16:55:16.294334 2888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:16.295830 2888 log.cc:824] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: Log is configured to *not* fsync() on all Append() calls
W20260812 16:55:16.309803 3098 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:55:16.309978 3098 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:55:16.294370 2899 master_runner.cc:186] Error getting master registration for 127.8.195.189:43181: OK, instance_id { permanent_uuid: "93d2772e869c416d89699fa4343c2611" instance_seqno: 1786553715526670 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:55:16.313666 2899 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:55:16.316886 2888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: No bootstrap required, opened a new log
I20260812 16:55:16.330169 2888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.330807 2888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:16.330997 2888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93d2772e869c416d89699fa4343c2611, State: Initialized, Role: FOLLOWER
I20260812 16:55:16.331797 2888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.334514 2888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:16.337570 3109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.338039 3109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 16:55:16.341879 2945 tablet.cc:2369] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:16.354384 2756 master_runner.cc:263] No leader master found from master 93d2772e869c416d89699fa4343c2611
I20260812 16:55:16.354575 2756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 16:55:16.360045 3135 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:55:16.360353 3135 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:55:16.461812 3109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:16.462220 3109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.463999 3109 leader_election.cc:288] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2afcd6b0af4c41f8a46ec5c385ec627e (127.8.195.190:45171), d0ba09d17f26479d833e02f5734ba7e2 (127.8.195.188:34607)
I20260812 16:55:16.465339 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93d2772e869c416d89699fa4343c2611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ba09d17f26479d833e02f5734ba7e2" is_pre_election: true
I20260812 16:55:16.466040 2986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93d2772e869c416d89699fa4343c2611 in term 0.
I20260812 16:55:16.466753 2796 leader_election.cc:302] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93d2772e869c416d89699fa4343c2611, d0ba09d17f26479d833e02f5734ba7e2; no voters:
I20260812 16:55:16.467238 3109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:55:16.468042 2689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93d2772e869c416d89699fa4343c2611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" is_pre_election: true
I20260812 16:55:16.468679 2689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93d2772e869c416d89699fa4343c2611 in term 0.
I20260812 16:55:16.469280 3109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:55:16.469422 3109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:16.474484 3109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.477969 2689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93d2772e869c416d89699fa4343c2611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e"
I20260812 16:55:16.478247 2689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:16.482280 2689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93d2772e869c416d89699fa4343c2611 in term 1.
I20260812 16:55:16.482954 2796 leader_election.cc:302] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2afcd6b0af4c41f8a46ec5c385ec627e, 93d2772e869c416d89699fa4343c2611; no voters:
I20260812 16:55:16.483960 3109 leader_election.cc:288] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [CANDIDATE]: Term 1 election: Requested vote from peers 2afcd6b0af4c41f8a46ec5c385ec627e (127.8.195.190:45171), d0ba09d17f26479d833e02f5734ba7e2 (127.8.195.188:34607)
I20260812 16:55:16.484529 3109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:16.484967 3109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [term 1 LEADER]: Becoming Leader. State: Replica: 93d2772e869c416d89699fa4343c2611, State: Running, Role: LEADER
I20260812 16:55:16.485800 3109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } }
I20260812 16:55:16.496048 2986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93d2772e869c416d89699fa4343c2611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ba09d17f26479d833e02f5734ba7e2"
I20260812 16:55:16.496300 2986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:16.499913 2986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93d2772e869c416d89699fa4343c2611 in term 1.
I20260812 16:55:16.503319 3142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93d2772e869c416d89699fa4343c2611. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.503795 3142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:16.515059 3146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260812 16:55:16.515385 2801 tablet.cc:2369] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:16.520377 3146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:16.552073 2986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [term 1 FOLLOWER]: Refusing update from remote peer 93d2772e869c416d89699fa4343c2611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:16.554442 3142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:16.562412 2689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [term 1 FOLLOWER]: Refusing update from remote peer 93d2772e869c416d89699fa4343c2611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:16.563453 3142 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:16.606871 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93d2772e869c416d89699fa4343c2611. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.607251 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:55:16.612526 3153 mvcc.cc:205] Tried to move back new op lower bound from 7317724022982356992 to 7317724022739443712. Current Snapshot: MvccSnapshot[applied={T|T < 7317724022982356992}]
I20260812 16:55:16.613193 3040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.613474 3040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:55:16.613835 3149 mvcc.cc:205] Tried to move back new op lower bound from 7317724022982356992 to 7317724022739443712. Current Snapshot: MvccSnapshot[applied={T|T < 7317724022982356992}]
I20260812 16:55:16.615706 3078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93d2772e869c416d89699fa4343c2611. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.616043 3078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:55:16.625125 3152 mvcc.cc:205] Tried to move back new op lower bound from 7317724022982356992 to 7317724022739443712. Current Snapshot: MvccSnapshot[applied={T|T < 7317724022982356992}]
I20260812 16:55:16.628657 3078 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.628959 3078 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:55:16.629297 3142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.629585 3142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:16.629940 3142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93d2772e869c416d89699fa4343c2611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2afcd6b0af4c41f8a46ec5c385ec627e" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45171 } } peers { permanent_uuid: "93d2772e869c416d89699fa4343c2611" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 43181 } } peers { permanent_uuid: "d0ba09d17f26479d833e02f5734ba7e2" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 34607 } } }
I20260812 16:55:16.630152 3142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:16.633445 3146 catalog_manager.cc:1347] Generated new cluster ID: b9eff1a265a6442d9bedd61ac37fa88c
I20260812 16:55:16.633553 3146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:16.685325 3146 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:55:16.690498 3146 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:16.722158 3146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93d2772e869c416d89699fa4343c2611: Generated new TSK 0
I20260812 16:55:16.723315 3146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:55:17.192032 3062 catalog_manager.cc:1259] Loaded cluster ID: b9eff1a265a6442d9bedd61ac37fa88c
I20260812 16:55:17.192286 3062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: loading cluster ID for follower catalog manager: success
I20260812 16:55:17.194885 3062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: acquiring CA information for follower catalog manager: success
I20260812 16:55:17.196753 3062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2afcd6b0af4c41f8a46ec5c385ec627e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:55:17.312320 3098 catalog_manager.cc:1259] Loaded cluster ID: b9eff1a265a6442d9bedd61ac37fa88c
I20260812 16:55:17.312449 3098 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: loading cluster ID for follower catalog manager: success
I20260812 16:55:17.316051 3098 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: acquiring CA information for follower catalog manager: success
I20260812 16:55:17.316329 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 2624
I20260812 16:55:17.317752 2733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:17.318804 3098 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d0ba09d17f26479d833e02f5734ba7e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:55:17.430019 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 2624
I20260812 16:55:17.462016 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 2756
I20260812 16:55:17.463186 2880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:17.574898 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 2756
I20260812 16:55:17.608282 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 2899
I20260812 16:55:17.609586 3015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:17.720288 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 2899
2026-08-12T16:55:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4975 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:55:17 dist-test-slave-z8x9 krb5kdc[3174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:55:17 dist-test-slave-z8x9 krb5kdc[3174](info): set up 2 sockets
Aug 12 16:55:17 dist-test-slave-z8x9 krb5kdc[3174](info): commencing operation
krb5kdc: starting...
W20260812 16:55:19.169557 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.395s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:55:19 dist-test-slave-z8x9 krb5kdc[3174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:55:19.489346 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40343
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:40343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:19.848793 3200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:19.849117 3200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:19.849347 3200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:19.857309 3200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:19.857432 3200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:19.857532 3200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:19.857632 3200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:19.857718 3200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:19.868119 3200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:40343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:19.869674 3200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:19.871049 3200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:19.883404 3209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:19.883492 3210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:19.883471 3212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:19.884588 3200 server_base.cc:1034] running on GCE node
I20260812 16:55:19.885635 3200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:19.887207 3200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:19.888463 3200 hybrid_clock.cc:648] HybridClock initialized: now 1786553719888422 us; error 54 us; skew 500 ppm
I20260812 16:55:19.888831 3200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:19 dist-test-slave-z8x9 krb5kdc[3174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:19.892238 3200 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:19.894047 3200 webserver.cc:460] Webserver started at http://127.8.195.190:42833/ using document root <none> and password file <none>
I20260812 16:55:19.894802 3200 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:19.894966 3200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:19.895319 3200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:19.898562 3200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "7eece481774e483493a18671cf4a017c"
format_stamp: "Formatted at 2026-08-12 16:55:19 on dist-test-slave-z8x9"
server_key: "7b23432a503446ab91e772f90fb8d1e3"
server_key_iv: "dd6bab01087064f575d5aaa3ce8b0c40"
server_key_version: "encryptionkey@0"
I20260812 16:55:19.899387 3200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "7eece481774e483493a18671cf4a017c"
format_stamp: "Formatted at 2026-08-12 16:55:19 on dist-test-slave-z8x9"
server_key: "7b23432a503446ab91e772f90fb8d1e3"
server_key_iv: "dd6bab01087064f575d5aaa3ce8b0c40"
server_key_version: "encryptionkey@0"
I20260812 16:55:19.905014 3200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 16:55:19.908932 3223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:19.910696 3200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:19.911016 3200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "7eece481774e483493a18671cf4a017c"
format_stamp: "Formatted at 2026-08-12 16:55:19 on dist-test-slave-z8x9"
server_key: "7b23432a503446ab91e772f90fb8d1e3"
server_key_iv: "dd6bab01087064f575d5aaa3ce8b0c40"
server_key_version: "encryptionkey@0"
I20260812 16:55:19.911267 3200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:19.930322 3200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:19.934929 3200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:19.935247 3200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:19.955451 3200 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:40343
I20260812 16:55:19.955489 3318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:40343 every 8 connection(s)
I20260812 16:55:19.957046 3200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:19.960875 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 3200
I20260812 16:55:19.961241 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:55:19.961905 3321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:19.962255 8974 external_mini_cluster.cc:1428] Setting key 510969007a1e6c81bbcd58d32592fbc9
I20260812 16:55:19.976212 3321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c: Bootstrap starting.
Aug 12 16:55:19 dist-test-slave-z8x9 krb5kdc[3174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:19.979971 3321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:19.981415 3321 log.cc:824] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:19.984858 3321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c: No bootstrap required, opened a new log
I20260812 16:55:19.988278 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:19.964481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.8.195.190:40343)
0812 16:55:19.965542 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:19.965555 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:19.965602 (+ 47us) server_negotiation.cc:407] Connection header received
0812 16:55:19.967645 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:19.967672 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:19.968136 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:19.968596 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:19.970380 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:19.971733 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:19.973727 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:19.974580 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:19.974809 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:19.979816 (+ 5007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:19.979825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:19.979841 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:19.979889 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:19.983809 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:19.984614 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:19.984621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:19.984627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:19.984832 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:19.985336 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:19.985343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:19.985346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:19.985915 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:19.986260 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:19.986756 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 16:55:19.987215 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":569,"threads_started":1}
I20260812 16:55:19.992720 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:55:19.994498 3321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } }
I20260812 16:55:19.994947 3321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:19.995065 3321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eece481774e483493a18671cf4a017c, State: Initialized, Role: FOLLOWER
I20260812 16:55:19.995807 3321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } }
I20260812 16:55:19.996093 3321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:19.996291 3321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:19.996510 3321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:19.998988 3321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } }
I20260812 16:55:19.999534 3321 leader_election.cc:302] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eece481774e483493a18671cf4a017c; no voters:
I20260812 16:55:20.000104 3321 leader_election.cc:288] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:55:20.000320 3329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:20.001047 3329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [term 1 LEADER]: Becoming Leader. State: Replica: 7eece481774e483493a18671cf4a017c, State: Running, Role: LEADER
I20260812 16:55:20.001650 3329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } }
I20260812 16:55:20.002461 3321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:20.005030 3330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eece481774e483493a18671cf4a017c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } } }
I20260812 16:55:20.004940 3331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eece481774e483493a18671cf4a017c. Latest consensus state: current_term: 1 leader_uuid: "7eece481774e483493a18671cf4a017c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eece481774e483493a18671cf4a017c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 40343 } } }
I20260812 16:55:20.005499 3330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:20.005505 3331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:20.009430 3344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:20.013994 3344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:20.022421 3344 catalog_manager.cc:1347] Generated new cluster ID: 1fc0e0195bd64ab2882b07599399e582
I20260812 16:55:20.022568 3344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:20.052304 3344 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:55:20.053825 3344 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:20.061316 3344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eece481774e483493a18671cf4a017c: Generated new TSK 0
I20260812 16:55:20.062219 3344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:55:20.107128 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 3200
I20260812 16:55:20.108647 3312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:20.211828 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 3200
2026-08-12T16:55:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2489 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:55:20 dist-test-slave-z8x9 krb5kdc[3370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:55:20 dist-test-slave-z8x9 krb5kdc[3370](info): set up 2 sockets
Aug 12 16:55:20 dist-test-slave-z8x9 krb5kdc[3370](info): commencing operation
krb5kdc: starting...
W20260812 16:55:21.645684 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:55:21 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:55:21.969705 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:22.320811 3393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:22.321130 3393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:22.321400 3393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:22.328692 3393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:22.328799 3393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:22.328923 3393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:22.329026 3393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:22.329095 3393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:22.339087 3393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:22.340497 3393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:22.341888 3393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:22.354422 3405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:22.354511 3410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:22.354465 3412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:22.355574 3393 server_base.cc:1034] running on GCE node
I20260812 16:55:22.356513 3393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:22.358100 3393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:22.359326 3393 hybrid_clock.cc:648] HybridClock initialized: now 1786553722359277 us; error 45 us; skew 500 ppm
I20260812 16:55:22.359705 3393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:22 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553722, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:22.363199 3393 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:22.364850 3393 webserver.cc:460] Webserver started at http://127.8.195.190:36061/ using document root <none> and password file <none>
I20260812 16:55:22.365643 3393 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:22.365756 3393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:22.366026 3393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:22.369231 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0"
format_stamp: "Formatted at 2026-08-12 16:55:22 on dist-test-slave-z8x9"
server_key: "6a3733fa8770d8e0fe351722766acf03"
server_key_iv: "13cd29a847ef7e38e333153e7dc2e345"
server_key_version: "encryptionkey@0"
I20260812 16:55:22.370041 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0"
format_stamp: "Formatted at 2026-08-12 16:55:22 on dist-test-slave-z8x9"
server_key: "6a3733fa8770d8e0fe351722766acf03"
server_key_iv: "13cd29a847ef7e38e333153e7dc2e345"
server_key_version: "encryptionkey@0"
I20260812 16:55:22.375427 3393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:55:22.379070 3424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:22.380462 3393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:55:22.380667 3393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0"
format_stamp: "Formatted at 2026-08-12 16:55:22 on dist-test-slave-z8x9"
server_key: "6a3733fa8770d8e0fe351722766acf03"
server_key_iv: "13cd29a847ef7e38e333153e7dc2e345"
server_key_version: "encryptionkey@0"
I20260812 16:55:22.380893 3393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:22.398909 3393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:22.403429 3393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:22.403752 3393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:22.424059 3393 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34793
I20260812 16:55:22.424084 3523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34793 every 8 connection(s)
I20260812 16:55:22.425607 3393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:22.429570 3524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:22.431329 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 3393
I20260812 16:55:22.431571 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:55:22.432276 8974 external_mini_cluster.cc:1428] Setting key 401d19d0ad5af2cad41f3d085c40e529
I20260812 16:55:22.443646 3524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap starting.
I20260812 16:55:22.447440 3524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Neither blocks nor log segments found. Creating new log.
Aug 12 16:55:22 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:22.448578 3524 log.cc:824] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:22.452327 3524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: No bootstrap required, opened a new log
I20260812 16:55:22.458719 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:22.434494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.8.195.190:34793)
0812 16:55:22.435350 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:22.435370 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:22.435422 (+ 52us) server_negotiation.cc:407] Connection header received
0812 16:55:22.437981 (+ 2559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:22.438016 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:22.438554 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:22.439049 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:22.440464 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:22.442606 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:22.444020 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:22.445039 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:22.445240 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:22.449614 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:22.449622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:22.449636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:22.449678 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:22.453915 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:22.454633 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:22.454643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:22.454653 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:22.454895 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:22.455261 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:22.455271 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:22.455276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:22.456043 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:22.456372 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:22.456723 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:55:22.457246 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":342,"threads_started":1}
I20260812 16:55:22.462491 3524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:22.462867 3524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:22.462942 3524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Initialized, Role: FOLLOWER
I20260812 16:55:22.463616 3524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:22.463929 3524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:22.464102 3524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:22.464305 3524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:22.466840 3524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:22.467486 3524 leader_election.cc:302] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c9c2b8ac3a4ee285933d5dc384d4b0; no voters:
I20260812 16:55:22.468104 3524 leader_election.cc:288] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:55:22.468289 3535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:22.469097 3535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 LEADER]: Becoming Leader. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Running, Role: LEADER
I20260812 16:55:22.469710 3535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:22.470436 3524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:22.472851 3538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:22.473218 3538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:22.473817 3539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c9c2b8ac3a4ee285933d5dc384d4b0. Latest consensus state: current_term: 1 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:22.474164 3539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:22.474205 3551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:22.485133 3551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:22.496294 3551 catalog_manager.cc:1347] Generated new cluster ID: 50a47809a1c748a38f9dd1ca24551fa0
I20260812 16:55:22.496454 3551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:22.524289 3551 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:55:22.526038 3551 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:22.536211 3551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Generated new TSK 0
I20260812 16:55:22.537261 3551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:55:22.610325 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:22.972519 3574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:22.972831 3574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:22.972987 3574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:22.980882 3574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:22.981011 3574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:22.981266 3574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:22.992204 3574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:22.993817 3574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:22.995276 3574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:23.009627 3588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:23.009557 3584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:23.009620 3583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:23.010476 3574 server_base.cc:1034] running on GCE node
I20260812 16:55:23.011209 3574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:23.012279 3574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:23.013533 3574 hybrid_clock.cc:648] HybridClock initialized: now 1786553723013435 us; error 88 us; skew 500 ppm
I20260812 16:55:23.013888 3574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:23 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:23.017366 3574 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:23.018885 3574 webserver.cc:460] Webserver started at http://127.8.195.129:40837/ using document root <none> and password file <none>
I20260812 16:55:23.019593 3574 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:23.019698 3574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:23.019973 3574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:23.023078 3574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "5b765f0bead3484392ce618472f8fc37"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "013356561a5882ec73ed557de3c102d1"
server_key_iv: "e1093e04f38ecb8dc5bfd33ef4a3a6bb"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.023839 3574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "5b765f0bead3484392ce618472f8fc37"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "013356561a5882ec73ed557de3c102d1"
server_key_iv: "e1093e04f38ecb8dc5bfd33ef4a3a6bb"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.029302 3574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 16:55:23.032862 3604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.034324 3574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:55:23.034498 3574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "5b765f0bead3484392ce618472f8fc37"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "013356561a5882ec73ed557de3c102d1"
server_key_iv: "e1093e04f38ecb8dc5bfd33ef4a3a6bb"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.034682 3574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:23.053591 3574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:23.057833 3574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:23.058115 3574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:23.059089 3574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:23.060474 3574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:23.060559 3574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.060650 3574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:23.060698 3574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.095149 3574 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:42387
I20260812 16:55:23.095191 3792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:42387 every 8 connection(s)
I20260812 16:55:23.096716 3574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:23.102166 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 3574
I20260812 16:55:23.102411 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:55:23.103294 8974 external_mini_cluster.cc:1428] Setting key 2b197c7c3072a8c659c77f57c9eb28fb
Aug 12 16:55:23 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:23.121943 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:23.099901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:50289 (local address 127.8.195.190:34793)
0812 16:55:23.100444 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:23.100451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:23.101552 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 16:55:23.104516 (+ 2964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:23.104521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:23.104622 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:23.104757 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:23.106680 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.107432 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.109160 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.109836 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.109993 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:23.114150 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:23.114160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:23.114168 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:23.114220 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:23.118087 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:23.119256 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:23.119262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:23.119264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:23.119409 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:23.119972 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:23.119976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:23.119977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:23.120297 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:23.120417 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:23.121459 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0812 16:55:23.121656 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":254,"threads_started":1}
I20260812 16:55:23.123543 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:23.100480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:50289)
0812 16:55:23.101374 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:23.101422 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:23.104008 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:23.105075 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:23.105085 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:23.105786 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:23.106470 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:23.106491 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.107626 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.107630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:23.108944 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:23.108958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.110320 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.110326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:23.110582 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:23.111400 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:23.111431 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:23.113845 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:23.118274 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:23.118287 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:23.118309 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:23.119063 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:23.119539 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:23.119546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:23.119552 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:23.119836 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:23.120433 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:23.120445 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:23.120967 (+ 522us) client_negotiation.cc:769] Sending connection context
0812 16:55:23.121319 (+ 352us) client_negotiation.cc:240] Negotiation successful
0812 16:55:23.121959 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":304,"threads_started":1}
I20260812 16:55:23.125478 3793 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:23.125824 3793 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:23.126595 3793 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:23.129065 3452 ts_manager.cc:194] Registered new tserver with Master: 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387)
I20260812 16:55:23.131462 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:50289
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:55:23.174710 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:23.527288 3804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:23.527663 3804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:23.527854 3804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:23.535343 3804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:23.535533 3804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:23.535735 3804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:23.546485 3804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:23.547957 3804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:23.549510 3804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:23.567898 3816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:23.567991 3815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:23.570457 3818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:23.571897 3804 server_base.cc:1034] running on GCE node
I20260812 16:55:23.572745 3804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:23.573798 3804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:23.575058 3804 hybrid_clock.cc:648] HybridClock initialized: now 1786553723575001 us; error 42 us; skew 500 ppm
I20260812 16:55:23.575479 3804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:23 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:23.578882 3804 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:23.580470 3804 webserver.cc:460] Webserver started at http://127.8.195.130:43679/ using document root <none> and password file <none>
I20260812 16:55:23.581281 3804 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:23.581399 3804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:23.581663 3804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:23.584985 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "b9aa68e6863446148d24927219d964bc"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "64e8f318d93526412b9a6cb3a956ff07"
server_key_iv: "19b144f6f93496115b43e76113946020"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.585891 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "b9aa68e6863446148d24927219d964bc"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "64e8f318d93526412b9a6cb3a956ff07"
server_key_iv: "19b144f6f93496115b43e76113946020"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.591652 3804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 16:55:23.595530 3830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.597259 3804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:55:23.597481 3804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b9aa68e6863446148d24927219d964bc"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "64e8f318d93526412b9a6cb3a956ff07"
server_key_iv: "19b144f6f93496115b43e76113946020"
server_key_version: "encryptionkey@0"
I20260812 16:55:23.597697 3804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:23.609890 3804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:23.614763 3804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:23.615041 3804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:23.616058 3804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:23.617475 3804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:23.617552 3804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.617638 3804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:23.617682 3804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:23.653520 3804 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:45171
I20260812 16:55:23.653568 4050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:45171 every 8 connection(s)
I20260812 16:55:23.654909 3804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:23.659111 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 3804
I20260812 16:55:23.659340 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:55:23.659963 8974 external_mini_cluster.cc:1428] Setting key 4ec2d932f31f0c6b01b04699837cd52d
Aug 12 16:55:23 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:23.678985 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:23.657807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:45235 (local address 127.8.195.190:34793)
0812 16:55:23.658293 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:23.658297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:23.659524 (+ 1227us) server_negotiation.cc:407] Connection header received
0812 16:55:23.661998 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:23.662007 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:23.662095 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:23.662195 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:23.663952 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.665154 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.666783 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.667558 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.667713 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:23.672007 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:23.672017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:23.672024 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:23.672071 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:23.675367 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:23.676407 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:23.676411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:23.676413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:23.676558 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:23.677080 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:23.677083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:23.677085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:23.677465 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:23.677584 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:23.678523 (+ 939us) server_negotiation.cc:299] Negotiation successful
0812 16:55:23.678706 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":262,"threads_started":1}
I20260812 16:55:23.680783 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:23.658326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:45235)
0812 16:55:23.659332 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:23.659395 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:23.661642 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:23.662494 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:23.662503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:23.663064 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:23.663737 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:23.663757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.665344 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.665348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:23.666623 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:23.666635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:23.667790 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:23.667795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:23.668008 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:23.668794 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:23.668818 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:23.671713 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:23.675542 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:23.675552 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:23.675572 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:23.676257 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:23.676680 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:23.676688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:23.676694 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:23.676967 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:23.677596 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:23.677608 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:23.678094 (+ 486us) client_negotiation.cc:769] Sending connection context
0812 16:55:23.678441 (+ 347us) client_negotiation.cc:240] Negotiation successful
0812 16:55:23.679107 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":420,"threads_started":1}
I20260812 16:55:23.682448 4051 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:23.682814 4051 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:23.683637 4051 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:23.685650 3452 ts_manager.cc:194] Registered new tserver with Master: b9aa68e6863446148d24927219d964bc (127.8.195.130:45171)
I20260812 16:55:23.686956 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:45235
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:55:23.732882 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:24.093402 4059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:24.093708 4059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:24.093915 4059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:24.101933 4059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:24.102057 4059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:24.102274 4059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:24.112744 4059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:24.114297 4059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:24.115703 4059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:24.129894 4074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:24.129966 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:24.129954 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:24.130719 4059 server_base.cc:1034] running on GCE node
I20260812 16:55:24.131422 4059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:24.132423 4059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:24.133697 4059 hybrid_clock.cc:648] HybridClock initialized: now 1786553724133634 us; error 50 us; skew 500 ppm
I20260812 16:55:24.134085 4059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:55:24.134794 3793 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
Aug 12 16:55:24 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553724, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:24.137982 4059 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:24.139688 4059 webserver.cc:460] Webserver started at http://127.8.195.131:37791/ using document root <none> and password file <none>
I20260812 16:55:24.140383 4059 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:24.140492 4059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:24.140748 4059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:24.144014 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "f91cafd103b847b0b83029252b3e8bce"
format_stamp: "Formatted at 2026-08-12 16:55:24 on dist-test-slave-z8x9"
server_key: "de53f476c5f92ac4ba6d2de1fde4d4f6"
server_key_iv: "f7abdb234babc7d5707061be9d5791b7"
server_key_version: "encryptionkey@0"
I20260812 16:55:24.144770 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "f91cafd103b847b0b83029252b3e8bce"
format_stamp: "Formatted at 2026-08-12 16:55:24 on dist-test-slave-z8x9"
server_key: "de53f476c5f92ac4ba6d2de1fde4d4f6"
server_key_iv: "f7abdb234babc7d5707061be9d5791b7"
server_key_version: "encryptionkey@0"
I20260812 16:55:24.150754 4059 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.004s
I20260812 16:55:24.154672 4089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:24.156224 4059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:55:24.156440 4059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f91cafd103b847b0b83029252b3e8bce"
format_stamp: "Formatted at 2026-08-12 16:55:24 on dist-test-slave-z8x9"
server_key: "de53f476c5f92ac4ba6d2de1fde4d4f6"
server_key_iv: "f7abdb234babc7d5707061be9d5791b7"
server_key_version: "encryptionkey@0"
I20260812 16:55:24.156728 4059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:24.179337 4059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:24.184814 4059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:24.185220 4059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:24.186429 4059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:24.187891 4059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:24.188001 4059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:24.188158 4059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:24.188236 4059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:24.224022 4059 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:32875
I20260812 16:55:24.224087 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:32875 every 8 connection(s)
I20260812 16:55:24.225485 4059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:24.235800 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 4059
I20260812 16:55:24.236024 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:55:24.236582 8974 external_mini_cluster.cc:1428] Setting key f479de5cefd300ee904707cbd7cefedc
Aug 12 16:55:24 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553724, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:24.246222 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.228122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:45005 (local address 127.8.195.190:34793)
0812 16:55:24.228539 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.228544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.229574 (+ 1030us) server_negotiation.cc:407] Connection header received
0812 16:55:24.231401 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.231407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.231511 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.231647 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:24.233143 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.233970 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.235530 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.235949 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.236098 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.240460 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:24.240468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:24.240473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:24.240521 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:24.243389 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.244134 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.244138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.244140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.244255 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.244631 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.244634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.244636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.244908 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:24.245023 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.245777 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.245957 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":200,"threads_started":1}
I20260812 16:55:24.247226 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.228650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:45005)
0812 16:55:24.229419 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.229457 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.231131 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.231887 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.231895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.232409 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:24.232963 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.232980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.234140 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.234144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.235373 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.235386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.236206 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.236213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.236469 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.237273 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:24.237297 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:24.240215 (+ 2918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:24.243541 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:24.243548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:24.243561 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:24.244029 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:24.244351 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:24.244354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:24.244356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:24.244536 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:24.245013 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:24.245019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:24.245434 (+ 415us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.245687 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.246089 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":274,"threads_started":1}
I20260812 16:55:24.248637 4316 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:24.249090 4316 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:24.249850 4316 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:24.251480 3452 ts_manager.cc:194] Registered new tserver with Master: f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:24.252444 3452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:45005
I20260812 16:55:24.253566 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:55:24.272208 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.258621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.8.195.190:34793)
0812 16:55:24.258857 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.258866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.259212 (+ 346us) server_negotiation.cc:407] Connection header received
0812 16:55:24.259436 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.259448 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.259513 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.259619 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:24.261129 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.262001 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.263424 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.263839 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.263987 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.266641 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:24.266647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:24.266653 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:24.266693 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:24.269321 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.270003 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.270010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.270013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.270205 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.270616 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.270621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.270624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.270973 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:24.271255 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.271632 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.271884 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:55:24.276736 3450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:55:24.284235 3450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:55:24.316213 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.297243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:45171 (local address 127.0.0.1:50610)
0812 16:55:24.305406 (+ 8163us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.305450 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.305640 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.306819 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.306830 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.306871 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:24.307464 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.307483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.309819 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.309824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.313033 (+ 3209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.313053 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.315105 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.315115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.315403 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.315452 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.315623 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.315834 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7746,"thread_start_us":16119,"threads_started":1}
I20260812 16:55:24.316282 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.297492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.8.195.130:45171)
0812 16:55:24.304268 (+ 6776us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.304280 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.305856 (+ 1576us) server_negotiation.cc:407] Connection header received
0812 16:55:24.305967 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.305978 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.306266 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.306473 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:24.307720 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.309646 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.313234 (+ 3588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.314916 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.315040 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.315254 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.315702 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.315925 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6590,"thread_start_us":6436,"threads_started":1}
I20260812 16:55:24.320222 3920 tablet_service.cc:1467] Processing CreateTablet for tablet cc6094ef425c4fe39bfae97d3404461b (DEFAULT_TABLE table=test-table [id=be72d93289ac44fcac7834a5f8455ecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:24.321914 3920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6094ef425c4fe39bfae97d3404461b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:24.323374 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.306056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:32875 (local address 127.0.0.1:47904)
0812 16:55:24.313652 (+ 7596us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.313670 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.313794 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.314683 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.314689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.314736 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:24.315388 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.315408 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.317514 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.317523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.320708 (+ 3185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.320721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.322393 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.322399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.322590 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.322635 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.322755 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.322920 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7466,"mutex_wait_us":190,"spinlock_wait_cycles":9344,"thread_start_us":6351,"threads_started":1}
I20260812 16:55:24.323392 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.306322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.8.195.131:32875)
0812 16:55:24.313255 (+ 6933us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.313267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.313795 (+ 528us) server_negotiation.cc:407] Connection header received
0812 16:55:24.313954 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.313961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.314150 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.314369 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:24.315651 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.317332 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.320914 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.322188 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.322340 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.322591 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.322829 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.323007 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6648,"thread_start_us":6470,"threads_started":1}
I20260812 16:55:24.323501 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.298151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.8.195.129:42387)
0812 16:55:24.304623 (+ 6472us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.304629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.312936 (+ 8307us) server_negotiation.cc:407] Connection header received
0812 16:55:24.313820 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.313832 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.314152 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.314380 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:24.315722 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.317301 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.320434 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.321985 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.322127 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.322361 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.323000 (+ 639us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.323180 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6317,"thread_start_us":6143,"threads_started":1}
I20260812 16:55:24.324236 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.297462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35446)
0812 16:55:24.312753 (+ 15291us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.312780 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.313599 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.314683 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.314689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.314729 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:24.315517 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.315529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.317455 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.317463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.320225 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.320243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.322521 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.322531 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.322759 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.322801 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.323714 (+ 913us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.323852 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15128,"mutex_wait_us":237,"spinlock_wait_cycles":1280,"thread_start_us":7379,"threads_started":1}
I20260812 16:55:24.327798 4184 tablet_service.cc:1467] Processing CreateTablet for tablet cc6094ef425c4fe39bfae97d3404461b (DEFAULT_TABLE table=test-table [id=be72d93289ac44fcac7834a5f8455ecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:24.328239 3686 tablet_service.cc:1467] Processing CreateTablet for tablet cc6094ef425c4fe39bfae97d3404461b (DEFAULT_TABLE table=test-table [id=be72d93289ac44fcac7834a5f8455ecc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:24.329649 4184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6094ef425c4fe39bfae97d3404461b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:24.329646 3686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6094ef425c4fe39bfae97d3404461b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:24.335079 4346 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Bootstrap starting.
I20260812 16:55:24.339375 4346 tablet_bootstrap.cc:654] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:24.340548 4346 log.cc:824] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:24.343327 4346 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: No bootstrap required, opened a new log
I20260812 16:55:24.343627 4346 ts_tablet_manager.cc:1397] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 16:55:24.345827 4349 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Bootstrap starting.
I20260812 16:55:24.346606 4348 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Bootstrap starting.
I20260812 16:55:24.350463 4348 tablet_bootstrap.cc:654] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:24.351748 4348 log.cc:824] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:24.352051 4349 tablet_bootstrap.cc:654] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:24.353397 4349 log.cc:824] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:24.353780 4346 raft_consensus.cc:348] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.354255 4346 raft_consensus.cc:374] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:24.354377 4346 raft_consensus.cc:729] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9aa68e6863446148d24927219d964bc, State: Initialized, Role: FOLLOWER
I20260812 16:55:24.355047 4346 consensus_queue.cc:260] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.355296 4348 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: No bootstrap required, opened a new log
I20260812 16:55:24.355567 4348 ts_tablet_manager.cc:1397] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 16:55:24.356348 4349 tablet_bootstrap.cc:492] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: No bootstrap required, opened a new log
I20260812 16:55:24.356648 4349 ts_tablet_manager.cc:1397] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.002s
I20260812 16:55:24.357967 4051 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
I20260812 16:55:24.358641 4346 ts_tablet_manager.cc:1428] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260812 16:55:24.364796 4348 raft_consensus.cc:348] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.365248 4348 raft_consensus.cc:374] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:24.365370 4348 raft_consensus.cc:729] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b765f0bead3484392ce618472f8fc37, State: Initialized, Role: FOLLOWER
I20260812 16:55:24.366008 4348 consensus_queue.cc:260] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.367071 4349 raft_consensus.cc:348] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.367522 4349 raft_consensus.cc:374] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:24.367609 4348 ts_tablet_manager.cc:1428] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Time spent starting tablet: real 0.012s user 0.004s sys 0.007s
I20260812 16:55:24.367630 4349 raft_consensus.cc:729] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91cafd103b847b0b83029252b3e8bce, State: Initialized, Role: FOLLOWER
I20260812 16:55:24.368235 4349 consensus_queue.cc:260] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.369942 4316 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
I20260812 16:55:24.370605 4349 ts_tablet_manager.cc:1428] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
W20260812 16:55:24.406981 4052 tablet.cc:2369] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:55:24.477478 4318 tablet.cc:2369] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:24.548074 4355 raft_consensus.cc:482] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:24.548380 4355 raft_consensus.cc:504] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.550144 4355 leader_election.cc:288] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:24.557225 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.550511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.8.195.130:59523)
0812 16:55:24.551223 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.551242 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.551398 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.551806 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.551810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.551852 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:24.552244 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.552277 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.553522 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.553526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.554960 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.554973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.556343 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.556352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.556490 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.556513 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.556620 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.556757 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":383,"threads_started":1}
I20260812 16:55:24.557274 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.550662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:59523 (local address 127.8.195.129:42387)
0812 16:55:24.550862 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.550866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.551339 (+ 473us) server_negotiation.cc:407] Connection header received
0812 16:55:24.551568 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.551577 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.551656 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.551780 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:24.552440 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.553405 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.555121 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.556061 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.556243 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.556334 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.556659 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.557004 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:55:24.557621 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.551498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:35203 (local address 127.8.195.131:32875)
0812 16:55:24.551725 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.551734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.552093 (+ 359us) server_negotiation.cc:407] Connection header received
0812 16:55:24.552335 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.552339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.552440 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.552538 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:24.553086 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.553955 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.555698 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.556563 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.556739 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.556849 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.557208 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.557345 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 16:55:24.557833 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.551313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:32875 (local address 127.8.195.130:35203)
0812 16:55:24.551987 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.552002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:24.552158 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:24.552570 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:24.552577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:24.552603 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:24.552923 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.552932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.554096 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.554102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.555545 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:24.555558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.556797 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.556804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:24.557014 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.557037 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:55:24.557159 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 16:55:24.557472 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":419,"threads_started":1}
I20260812 16:55:24.558816 3714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6094ef425c4fe39bfae97d3404461b" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b765f0bead3484392ce618472f8fc37" is_pre_election: true
I20260812 16:55:24.558964 4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6094ef425c4fe39bfae97d3404461b" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91cafd103b847b0b83029252b3e8bce" is_pre_election: true
I20260812 16:55:24.559418 3714 raft_consensus.cc:2454] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 0.
I20260812 16:55:24.559502 4231 raft_consensus.cc:2454] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 0.
I20260812 16:55:24.560174 3833 leader_election.cc:302] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce; no voters:
I20260812 16:55:24.560700 4355 raft_consensus.cc:2781] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:55:24.560854 4355 raft_consensus.cc:482] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:55:24.560977 4355 raft_consensus.cc:3037] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:24.563446 4355 raft_consensus.cc:504] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.564060 4355 leader_election.cc:288] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 election: Requested vote from peers 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:24.564525 3714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6094ef425c4fe39bfae97d3404461b" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b765f0bead3484392ce618472f8fc37"
I20260812 16:55:24.564627 4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6094ef425c4fe39bfae97d3404461b" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91cafd103b847b0b83029252b3e8bce"
I20260812 16:55:24.564728 3714 raft_consensus.cc:3037] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:24.564884 4231 raft_consensus.cc:3037] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:24.567186 3714 raft_consensus.cc:2454] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 1.
I20260812 16:55:24.567193 4231 raft_consensus.cc:2454] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 1.
I20260812 16:55:24.567603 3833 leader_election.cc:302] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37, b9aa68e6863446148d24927219d964bc; no voters:
I20260812 16:55:24.567914 4355 raft_consensus.cc:2781] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:24.568403 4355 raft_consensus.cc:686] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 1 LEADER]: Becoming Leader. State: Replica: b9aa68e6863446148d24927219d964bc, State: Running, Role: LEADER
I20260812 16:55:24.568987 4355 consensus_queue.cc:237] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:24.574223 3452 catalog_manager.cc:5495] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc reported cstate change: term changed from 0 to 1, leader changed from <none> to b9aa68e6863446148d24927219d964bc (127.8.195.130). New cstate: current_term: 1 leader_uuid: "b9aa68e6863446148d24927219d964bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
W20260812 16:55:24.599695 3798 tablet.cc:2369] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:24.605731 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.600281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.8.195.130:45171)
0812 16:55:24.600546 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.600554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.600582 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:55:24.600708 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.600714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.600805 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.600950 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:55:24.601407 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.602164 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.603648 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.604068 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.604166 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.604628 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:55:24.605142 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:55:24.605324 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.605428 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.605499 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 16:55:24.614650 3714 raft_consensus.cc:1261] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Refusing update from remote peer b9aa68e6863446148d24927219d964bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:24.614650 4231 raft_consensus.cc:1261] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Refusing update from remote peer b9aa68e6863446148d24927219d964bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:24.615784 4355 consensus_queue.cc:1035] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:24.616144 4362 consensus_queue.cc:1035] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:24.642737 3450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:24.643085 3450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:24.647198 3450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Sending DeleteTablet for 3 replicas of tablet cc6094ef425c4fe39bfae97d3404461b
I20260812 16:55:24.648433 3920 tablet_service.cc:1514] Processing DeleteTablet for tablet cc6094ef425c4fe39bfae97d3404461b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:24 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:50610
I20260812 16:55:24.648567 4184 tablet_service.cc:1514] Processing DeleteTablet for tablet cc6094ef425c4fe39bfae97d3404461b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:24 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:47904
I20260812 16:55:24.648729 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 3574
I20260812 16:55:24.649730 3686 tablet_service.cc:1514] Processing DeleteTablet for tablet cc6094ef425c4fe39bfae97d3404461b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:24 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35446
I20260812 16:55:24.659114 4391 tablet_replica.cc:330] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: stopping tablet replica
I20260812 16:55:24.659571 4391 raft_consensus.cc:2229] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:55:24.660260 4391 raft_consensus.cc:2258] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:24.662621 4391 ts_tablet_manager.cc:1905] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:24.670454 4391 ts_tablet_manager.cc:1918] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:24.670670 4391 log.cc:1196] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/cc6094ef425c4fe39bfae97d3404461b
Aug 12 16:55:24 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:55:24.671396 4391 ts_tablet_manager.cc:1939] T cc6094ef425c4fe39bfae97d3404461b P b9aa68e6863446148d24927219d964bc: Deleting consensus metadata
I20260812 16:55:24.673543 3428 catalog_manager.cc:4841] TS b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): tablet cc6094ef425c4fe39bfae97d3404461b (table test-table [id=be72d93289ac44fcac7834a5f8455ecc]) successfully deleted
I20260812 16:55:24.674665 4396 tablet_replica.cc:330] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: stopping tablet replica
I20260812 16:55:24.675222 4396 raft_consensus.cc:2229] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:24.675875 4396 raft_consensus.cc:2258] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:24.676702 4394 tablet_replica.cc:330] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: stopping tablet replica
I20260812 16:55:24.677224 4394 raft_consensus.cc:2229] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:24.677788 4394 raft_consensus.cc:2258] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:24.679948 4396 ts_tablet_manager.cc:1905] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:24.680465 4394 ts_tablet_manager.cc:1905] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:24.682467 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.650423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.8.195.129:42387)
0812 16:55:24.650607 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.650613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.658494 (+ 7881us) server_negotiation.cc:407] Connection header received
0812 16:55:24.659137 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.659146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.659247 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.659326 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:24.661788 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.662975 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.665044 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.665721 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.665821 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.672627 (+ 6806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:24.672652 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:24.672672 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:24.672722 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:24.677237 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.678397 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.678408 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.678411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.678600 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.679139 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.679145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.679149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.679661 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:24.680030 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.681482 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.681996 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:55:24.683279 3785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:24.691224 4394 ts_tablet_manager.cc:1918] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:24.691427 4394 log.cc:1196] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/cc6094ef425c4fe39bfae97d3404461b
I20260812 16:55:24.692013 4394 ts_tablet_manager.cc:1939] T cc6094ef425c4fe39bfae97d3404461b P f91cafd103b847b0b83029252b3e8bce: Deleting consensus metadata
I20260812 16:55:24.694175 3425 catalog_manager.cc:4841] TS f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): tablet cc6094ef425c4fe39bfae97d3404461b (table test-table [id=be72d93289ac44fcac7834a5f8455ecc]) successfully deleted
I20260812 16:55:24.783327 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 3574
W20260812 16:55:24.807833 3425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:42387 (error 108)
I20260812 16:55:24.808670 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 3804
I20260812 16:55:24.810215 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.809647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35460)
0812 16:55:24.809935 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.810075 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260812 16:55:24.810542 3425 catalog_manager.cc:4584] TS 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc6094ef425c4fe39bfae97d3404461b: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Aug 12 16:55:24 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:55:24.834714 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.809926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.8.195.130:45171)
0812 16:55:24.810115 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.810121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.810142 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:55:24.810208 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.810215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.810293 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.810409 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:24.812372 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.813506 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.815585 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.816104 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.816230 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:24.828039 (+ 11809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:24.828066 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:24.828083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:24.828129 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:24.831533 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.832507 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.832515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.832517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.832674 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:24.833114 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:24.833118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:24.833120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:24.833524 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:24.833814 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:24.834165 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:55:24.834355 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:55:24.835630 4042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:24.864248 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.863710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35474)
0812 16:55:24.863982 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.864096 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 16:55:24.865432 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.865015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35486)
0812 16:55:24.865196 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.865300 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 16:55:24.926234 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.925725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35494)
0812 16:55:24.925970 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.926058 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 16:55:24.927379 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.927004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35510)
0812 16:55:24.927171 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:24.927251 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 16:55:24.955837 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 3804
I20260812 16:55:24.985347 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 4059
Aug 12 16:55:24 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:55:25.007364 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:24.986156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.8.195.131:32875)
0812 16:55:24.986327 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:24.986332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:24.986351 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:55:24.986525 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:24.986529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:24.986608 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:24.986698 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:24.987946 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.988668 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.990091 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:24.990495 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:24.990604 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:25.000666 (+ 10062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:25.000704 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:25.000722 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:25.000773 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:25.003824 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:25.004720 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:25.004729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:25.004732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:25.004921 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:25.005622 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:25.005626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:25.005629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:25.006064 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:25.006366 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:25.006859 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 16:55:25.007057 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:55:25.008184 4307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:25.008507 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:25.008011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35526)
0812 16:55:25.008214 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:25.008296 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 16:55:25.009791 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:25.009285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35534)
0812 16:55:25.009529 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:25.009629 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260812 16:55:25.100550 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 4059
I20260812 16:55:25.128167 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 3393
I20260812 16:55:25.129522 3513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:25.244050 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 3393
I20260812 16:55:25.280305 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--webserver_interface=127.8.195.190
--webserver_port=36061
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:25.663183 4409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:25.663533 4409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:25.663687 4409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:25.671455 4409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:25.671648 4409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:25.671754 4409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:25.671875 4409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:25.671931 4409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:25.682493 4409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=36061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:25.683907 4409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:25.685370 4409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:25.713579 4419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:25.713546 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:25.713505 4418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:25.714507 4409 server_base.cc:1034] running on GCE node
I20260812 16:55:25.715567 4409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:25.717442 4409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:25.718708 4409 hybrid_clock.cc:648] HybridClock initialized: now 1786553725718650 us; error 51 us; skew 500 ppm
I20260812 16:55:25.719149 4409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:25 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553725, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:25.723045 4409 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:25.725054 4409 webserver.cc:460] Webserver started at http://127.8.195.190:36061/ using document root <none> and password file <none>
I20260812 16:55:25.725919 4409 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:25.726101 4409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:25.732352 4409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 16:55:25.736016 4439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:25.737705 4409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:25.737876 4409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0"
format_stamp: "Formatted at 2026-08-12 16:55:22 on dist-test-slave-z8x9"
server_key: "6a3733fa8770d8e0fe351722766acf03"
server_key_iv: "13cd29a847ef7e38e333153e7dc2e345"
server_key_version: "encryptionkey@0"
I20260812 16:55:25.739101 4409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:25.766275 4409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:25.771225 4409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:25.771479 4409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:25.792685 4409 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34793
I20260812 16:55:25.792737 4542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34793 every 8 connection(s)
I20260812 16:55:25.794443 4409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:25.800474 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 4409
I20260812 16:55:25.801527 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:42387
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=40837
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:25.803301 4543 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:55:25.808955 4543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap starting.
I20260812 16:55:25.831092 4543 log.cc:824] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:25.844573 4543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:55:25.845250 4543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap complete.
I20260812 16:55:25.855142 4543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:25.856599 4543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Initialized, Role: FOLLOWER
I20260812 16:55:25.857465 4543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:25.857779 4543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:25.857914 4543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:25.858119 4543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:55:25.862764 4543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:25.863462 4543 leader_election.cc:302] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c9c2b8ac3a4ee285933d5dc384d4b0; no voters:
I20260812 16:55:25.864207 4543 leader_election.cc:288] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:55:25.864456 4548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:55:25.866982 4543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:25.871759 4548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 LEADER]: Becoming Leader. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Running, Role: LEADER
I20260812 16:55:25.872761 4548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:25.880009 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c9c2b8ac3a4ee285933d5dc384d4b0. Latest consensus state: current_term: 2 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:25.880455 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:25.881469 4549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:25.881791 4549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:25.895274 4581 catalog_manager.cc:1259] Loaded cluster ID: 50a47809a1c748a38f9dd1ca24551fa0
I20260812 16:55:25.895430 4581 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: loading cluster ID for follower catalog manager: success
W20260812 16:55:25.897058 4581 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260812 16:55:25.897253 4581 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260812 16:55:25.897397 4584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:25.902750 4584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:25.904060 4584 catalog_manager.cc:1259] Loaded cluster ID: 50a47809a1c748a38f9dd1ca24551fa0
I20260812 16:55:25.904179 4584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:25.906942 4584 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 16:55:25.924001 4584 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 16:55:25.934417 4584 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:25.936201 4584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Loaded TSK: 0
I20260812 16:55:25.937837 4584 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:55:26.186373 4545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:26.186692 4545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:26.186841 4545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:26.194593 4545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:26.194723 4545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:26.194901 4545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:26.205446 4545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:42387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=40837
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:26.206871 4545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:26.208338 4545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:26.225556 4604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:26.225796 4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:26.226452 4597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:26.227077 4545 server_base.cc:1034] running on GCE node
I20260812 16:55:26.228067 4545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:26.229740 4545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:26.231050 4545 hybrid_clock.cc:648] HybridClock initialized: now 1786553726230962 us; error 65 us; skew 500 ppm
I20260812 16:55:26.231523 4545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:26 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:26.236687 4545 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:26.239012 4545 webserver.cc:460] Webserver started at http://127.8.195.129:40837/ using document root <none> and password file <none>
I20260812 16:55:26.240180 4545 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:26.240377 4545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:26.248934 4545 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:55:26.253330 4615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:26.255331 4545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 16:55:26.255558 4545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "5b765f0bead3484392ce618472f8fc37"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "013356561a5882ec73ed557de3c102d1"
server_key_iv: "e1093e04f38ecb8dc5bfd33ef4a3a6bb"
server_key_version: "encryptionkey@0"
I20260812 16:55:26.257304 4545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:26.298316 4545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:26.304883 4545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:26.305296 4545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:26.306696 4545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:26.309525 4628 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 16:55:26.315275 4628 ts_tablet_manager.cc:1852] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 16:55:26.316471 4628 ts_tablet_manager.cc:1905] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:26.324700 4628 ts_tablet_manager.cc:1918] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:26.324985 4628 log.cc:1196] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/cc6094ef425c4fe39bfae97d3404461b
I20260812 16:55:26.325635 4628 ts_tablet_manager.cc:1939] T cc6094ef425c4fe39bfae97d3404461b P 5b765f0bead3484392ce618472f8fc37: Deleting consensus metadata
I20260812 16:55:26.326126 4545 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 16:55:26.326270 4545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.000s sys 0.001s
I20260812 16:55:26.326414 4545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:26.326498 4545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:26.365603 4545 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:42387
I20260812 16:55:26.365659 4855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:42387 every 8 connection(s)
I20260812 16:55:26.367134 4545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:26.377302 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 4545
I20260812 16:55:26.378197 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:45171
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=43679
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:26 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:26.396836 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:26.370280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:48749 (local address 127.8.195.190:34793)
0812 16:55:26.370906 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:26.370919 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:26.371602 (+ 683us) server_negotiation.cc:407] Connection header received
0812 16:55:26.374717 (+ 3115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:26.374731 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:26.375107 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:26.375417 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:26.377141 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.379330 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.381152 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.382342 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.382572 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:26.386712 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:26.386728 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:26.386754 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:26.386819 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:26.391325 (+ 4506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:26.392581 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:26.392590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:26.392596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:26.392777 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:26.393391 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:26.393395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:26.393397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:26.393966 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:26.394122 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:26.395477 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0812 16:55:26.395846 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":275,"threads_started":1}
I20260812 16:55:26.397210 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:26.370466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:48749)
0812 16:55:26.371388 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:26.371450 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:26.373966 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:26.375588 (+ 1622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:26.375596 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:26.376231 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:26.376839 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:26.376859 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.379546 (+ 2687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.379551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:26.380972 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:26.380988 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.382615 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.382621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:26.382851 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:26.383656 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:26.383698 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:26.386340 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:26.391558 (+ 5218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:26.391571 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:26.391591 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:26.392405 (+ 814us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:26.392927 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:26.392936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:26.392942 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:26.393258 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:26.394140 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:26.394151 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:26.394712 (+ 561us) client_negotiation.cc:769] Sending connection context
0812 16:55:26.395069 (+ 357us) client_negotiation.cc:240] Negotiation successful
0812 16:55:26.395737 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":396,"threads_started":1}
I20260812 16:55:26.400022 4857 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:26.400391 4857 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:26.401226 4857 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:26.403545 4466 ts_manager.cc:194] Registered new tserver with Master: 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387)
I20260812 16:55:26.405069 4466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:48749
W20260812 16:55:26.735601 4865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:26.735957 4865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:26.736138 4865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:26.743857 4865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:26.744005 4865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:26.744199 4865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:26.754030 4865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:45171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=43679
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:26.755446 4865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:26.756896 4865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:26.771081 4873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:26.771189 4878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:26.771353 4875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:26.773219 4865 server_base.cc:1034] running on GCE node
I20260812 16:55:26.774204 4865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:26.775381 4865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:26.776650 4865 hybrid_clock.cc:648] HybridClock initialized: now 1786553726776593 us; error 63 us; skew 500 ppm
I20260812 16:55:26.777029 4865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:26 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:26.781095 4865 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:26.783002 4865 webserver.cc:460] Webserver started at http://127.8.195.130:43679/ using document root <none> and password file <none>
I20260812 16:55:26.783775 4865 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:26.783957 4865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:26.789728 4865 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 16:55:26.792826 4891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:26.794391 4865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:26.794595 4865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b9aa68e6863446148d24927219d964bc"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "64e8f318d93526412b9a6cb3a956ff07"
server_key_iv: "19b144f6f93496115b43e76113946020"
server_key_version: "encryptionkey@0"
I20260812 16:55:26.795621 4865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:26.811300 4865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:26.815521 4865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:26.815821 4865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:26.816867 4865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:26.818353 4865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:26.818496 4865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:26.818624 4865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:26.818715 4865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:26.852897 4865 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:45171
I20260812 16:55:26.852989 5110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:45171 every 8 connection(s)
I20260812 16:55:26.854420 4865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:26.864382 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 4865
I20260812 16:55:26.865082 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:32875
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=37791
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:55:26 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553726, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:26.877398 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:26.857013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:57843 (local address 127.8.195.190:34793)
0812 16:55:26.857246 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:26.857252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:26.858353 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 16:55:26.860624 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:26.860642 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:26.860792 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:26.860924 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:26.862982 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.863858 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.865885 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.866345 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.866488 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:26.870693 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:26.870699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:26.870703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:26.870738 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:26.873936 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:26.874847 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:26.874853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:26.874857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:26.875002 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:26.875472 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:26.875477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:26.875481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:26.875818 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:26.875986 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:26.876748 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 16:55:26.877006 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:55:26.878427 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:26.857495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:57843)
0812 16:55:26.858175 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:26.858218 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:26.860294 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:26.861250 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:26.861262 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:26.862060 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:26.862790 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:26.862810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.864070 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.864076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:26.865713 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:26.865728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:26.866601 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:26.866610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:26.866863 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:26.867673 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:26.867697 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:26.870424 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:26.874084 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:26.874094 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:26.874112 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:26.874708 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:26.875118 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:26.875123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:26.875127 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:26.875339 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:26.875953 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:26.875961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:26.876341 (+ 380us) client_negotiation.cc:769] Sending connection context
0812 16:55:26.876624 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 16:55:26.877235 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":268,"threads_started":1}
I20260812 16:55:26.880033 5111 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:26.880358 5111 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:26.881093 5111 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:26.882720 4466 ts_manager.cc:194] Registered new tserver with Master: b9aa68e6863446148d24927219d964bc (127.8.195.130:45171)
I20260812 16:55:26.884025 4466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:57843
W20260812 16:55:27.226956 5116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:27.227382 5116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:27.227488 5116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:27.234860 5116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:27.234987 5116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:27.235100 5116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:27.245226 5116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:32875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=37791
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:27.246577 5116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:27.247982 5116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:27.261834 5128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:27.261880 5124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:27.261919 5125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:27.263105 5116 server_base.cc:1034] running on GCE node
I20260812 16:55:27.263805 5116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:27.264962 5116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:27.266258 5116 hybrid_clock.cc:648] HybridClock initialized: now 1786553727266161 us; error 69 us; skew 500 ppm
I20260812 16:55:27.266644 5116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:27 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553727, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:27.270311 5116 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:27.271962 5116 webserver.cc:460] Webserver started at http://127.8.195.131:37791/ using document root <none> and password file <none>
I20260812 16:55:27.272827 5116 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:27.273005 5116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:27.279134 5116 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 16:55:27.282282 5142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:27.283710 5116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:55:27.283913 5116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f91cafd103b847b0b83029252b3e8bce"
format_stamp: "Formatted at 2026-08-12 16:55:24 on dist-test-slave-z8x9"
server_key: "de53f476c5f92ac4ba6d2de1fde4d4f6"
server_key_iv: "f7abdb234babc7d5707061be9d5791b7"
server_key_version: "encryptionkey@0"
I20260812 16:55:27.284974 5116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:27.326462 5116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:27.331322 5116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:27.331632 5116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:27.332768 5116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:27.334317 5116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:27.334450 5116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:27.334589 5116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:27.334681 5116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:27.373584 5116 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:32875
I20260812 16:55:27.373656 5369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:32875 every 8 connection(s)
I20260812 16:55:27.375173 5116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:27.380373 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 5116
Aug 12 16:55:27 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553727, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:27.401532 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.381415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.8.195.190:34793)
0812 16:55:27.382091 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.382101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.384370 (+ 2269us) server_negotiation.cc:407] Connection header received
0812 16:55:27.384507 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.384513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.384593 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.384714 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:27.386681 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.387956 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.389809 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.390616 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.390774 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.394187 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:27.394194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:27.394199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:27.394258 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:27.397823 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.398838 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.398842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.398844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.399049 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.399580 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.399585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.399588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.399961 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:27.400185 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.400790 (+ 605us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.401094 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":260,"threads_started":1}
I20260812 16:55:27.401662 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.378141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:41099 (local address 127.8.195.190:34793)
0812 16:55:27.378625 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.378630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.379564 (+ 934us) server_negotiation.cc:407] Connection header received
0812 16:55:27.381804 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.381809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.381883 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.382005 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:27.383925 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.385017 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.386798 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.387367 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.387481 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.392871 (+ 5390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:27.392876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:27.392879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:27.392917 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:27.396982 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.398179 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.398183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.398186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.398325 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.398961 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.398965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.398968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.399438 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:27.399645 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.400855 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.401257 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":373,"spinlock_wait_cycles":7040,"thread_start_us":266,"threads_started":1}
I20260812 16:55:27.402881 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.378610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:41099)
0812 16:55:27.379388 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.379428 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.381304 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.382283 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.382292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.382986 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:27.383626 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.383670 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.385218 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.385224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.386529 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.386544 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.387721 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.387731 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.388151 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.389220 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:27.389256 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:27.392584 (+ 3328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:27.397309 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:27.397320 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:27.397336 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:27.397977 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:27.398459 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:27.398466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:27.398470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:27.398803 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:27.399721 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:27.399734 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:27.400268 (+ 534us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.400698 (+ 430us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.401440 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":307,"threads_started":1}
I20260812 16:55:27.404838 5370 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34793
I20260812 16:55:27.405278 5370 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:27.406123 5370 heartbeater.cc:511] Master 127.8.195.190:34793 requested a full tablet report, sending...
I20260812 16:55:27.407824 4465 ts_manager.cc:194] Registered new tserver with Master: f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:27.408339 4857 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
I20260812 16:55:27.409271 4465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:41099
I20260812 16:55:27.418576 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:55:27.426038 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.419578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33778 (local address 127.8.195.190:34793)
0812 16:55:27.419771 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.419776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.420335 (+ 559us) server_negotiation.cc:407] Connection header received
0812 16:55:27.420536 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.420540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.420686 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.420823 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:55:27.421466 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.422202 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.423750 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.424182 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.424286 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.424859 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:55:27.425466 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:55:27.425590 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.425746 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.425819 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:55:27.427060 4466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:55:27.434969 4466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:55:27.471529 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.447778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:45171 (local address 127.0.0.1:50634)
0812 16:55:27.459940 (+ 12162us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.459961 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.460092 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.461694 (+ 1602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.461708 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.461759 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:27.462428 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.462447 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.465530 (+ 3083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.465537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.468168 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.468184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.470392 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.470399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.470617 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.470712 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.470886 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.471085 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12010,"mutex_wait_us":30,"thread_start_us":10699,"threads_started":1}
I20260812 16:55:27.472667 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.447840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50634 (local address 127.8.195.130:45171)
0812 16:55:27.460713 (+ 12873us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.460726 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.460770 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:55:27.460917 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.460930 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.461325 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.462748 (+ 1423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:27.463087 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.465350 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.468390 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.470135 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.470290 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.470497 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.472113 (+ 1616us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.472323 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12649,"thread_start_us":12511,"threads_started":1}
I20260812 16:55:27.473837 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.447653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.8.195.129:42387)
0812 16:55:27.461138 (+ 13485us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.461145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.461216 (+ 71us) server_negotiation.cc:407] Connection header received
0812 16:55:27.461338 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.461347 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.461627 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.461899 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:27.465900 (+ 4001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.467919 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.470862 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.472177 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.472342 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.472598 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.472957 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.473213 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13309,"thread_start_us":14664,"threads_started":1}
I20260812 16:55:27.474498 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.461384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:32875 (local address 127.0.0.1:47918)
0812 16:55:27.463040 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.463058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.463348 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.466628 (+ 3280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.466633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.466660 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:27.467217 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.467231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.469054 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.469062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.470829 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.470844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.473583 (+ 2739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.473592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.473773 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.473804 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.473897 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.474041 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1511,"mutex_wait_us":4,"thread_start_us":1338,"threads_started":1}
I20260812 16:55:27.474643 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.447294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35554)
0812 16:55:27.459105 (+ 11811us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.459169 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.459448 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.462218 (+ 2770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.462226 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.462267 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:27.465641 (+ 3374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.465660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.468426 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.468431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.470664 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.470677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.472402 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.472409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.472640 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.472678 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.474181 (+ 1503us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.474304 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11407,"mutex_wait_us":2921,"thread_start_us":12323,"threads_started":1}
I20260812 16:55:27.475277 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.460406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.8.195.131:32875)
0812 16:55:27.466088 (+ 5682us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.466094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.466124 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:55:27.466207 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.466214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.466417 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.466647 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:27.467459 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.468912 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.471242 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.472741 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.473175 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.473431 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.474760 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.474939 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5510,"thread_start_us":5335,"threads_started":1}
I20260812 16:55:27.477149 4981 tablet_service.cc:1467] Processing CreateTablet for tablet 8830e26591094f5fa146447af486a384 (DEFAULT_TABLE table=test-table [id=588ab6cefa1046499119161451a8f697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:27.478585 4981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830e26591094f5fa146447af486a384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:27.479233 4717 tablet_service.cc:1467] Processing CreateTablet for tablet 8830e26591094f5fa146447af486a384 (DEFAULT_TABLE table=test-table [id=588ab6cefa1046499119161451a8f697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:27.480624 4717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830e26591094f5fa146447af486a384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:27.482115 5244 tablet_service.cc:1467] Processing CreateTablet for tablet 8830e26591094f5fa146447af486a384 (DEFAULT_TABLE table=test-table [id=588ab6cefa1046499119161451a8f697]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:27.483580 5244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830e26591094f5fa146447af486a384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:27.492249 5400 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Bootstrap starting.
I20260812 16:55:27.497807 5400 tablet_bootstrap.cc:654] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:27.498677 5399 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Bootstrap starting.
I20260812 16:55:27.499603 5400 log.cc:824] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:27.502668 5399 tablet_bootstrap.cc:654] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:27.503019 5400 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: No bootstrap required, opened a new log
I20260812 16:55:27.503289 5400 ts_tablet_manager.cc:1397] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 16:55:27.503873 5399 log.cc:824] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:27.504668 5401 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Bootstrap starting.
I20260812 16:55:27.507503 5399 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: No bootstrap required, opened a new log
I20260812 16:55:27.507809 5399 ts_tablet_manager.cc:1397] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 16:55:27.507929 5401 tablet_bootstrap.cc:654] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:27.509054 5401 log.cc:824] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:27.512020 5401 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: No bootstrap required, opened a new log
I20260812 16:55:27.512363 5401 ts_tablet_manager.cc:1397] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 16:55:27.514321 5400 raft_consensus.cc:348] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.514810 5400 raft_consensus.cc:374] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:27.514919 5400 raft_consensus.cc:729] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b765f0bead3484392ce618472f8fc37, State: Initialized, Role: FOLLOWER
I20260812 16:55:27.515650 5400 consensus_queue.cc:260] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.517647 5400 ts_tablet_manager.cc:1428] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260812 16:55:27.519093 5399 raft_consensus.cc:348] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.519565 5399 raft_consensus.cc:374] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:27.519663 5399 raft_consensus.cc:729] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9aa68e6863446148d24927219d964bc, State: Initialized, Role: FOLLOWER
I20260812 16:55:27.520344 5399 consensus_queue.cc:260] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.521860 5111 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
I20260812 16:55:27.521732 5401 raft_consensus.cc:348] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.522176 5401 raft_consensus.cc:374] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:27.522284 5401 raft_consensus.cc:729] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91cafd103b847b0b83029252b3e8bce, State: Initialized, Role: FOLLOWER
I20260812 16:55:27.522609 5399 ts_tablet_manager.cc:1428] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260812 16:55:27.522867 5401 consensus_queue.cc:260] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.524937 5370 heartbeater.cc:503] Master 127.8.195.190:34793 was elected leader, sending a full tablet report...
I20260812 16:55:27.525698 5401 ts_tablet_manager.cc:1428] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
W20260812 16:55:27.606767 5112 tablet.cc:2369] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:55:27.619915 4861 tablet.cc:2369] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:55:27.627018 5372 tablet.cc:2369] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:27.661023 5407 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:27.661418 5407 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.663152 5407 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:27.670652 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.663625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:38367 (local address 127.8.195.129:42387)
0812 16:55:27.663829 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.663834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.664424 (+ 590us) server_negotiation.cc:407] Connection header received
0812 16:55:27.664802 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.664808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.664894 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.665015 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:27.665742 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.666506 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.668813 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.669678 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.669869 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.669951 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.670264 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.670394 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 16:55:27.670861 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.663522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.8.195.130:38367)
0812 16:55:27.664299 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.664334 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.664556 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.665276 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.665279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.665301 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:27.665592 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.665600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.666650 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.666656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.668647 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.668664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.669912 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.669919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.670087 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.670111 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.670225 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.670366 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":28,"thread_start_us":396,"threads_started":1}
I20260812 16:55:27.671366 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.664714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:54963 (local address 127.8.195.131:32875)
0812 16:55:27.664956 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.664962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.665227 (+ 265us) server_negotiation.cc:407] Connection header received
0812 16:55:27.665535 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.665541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.665612 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.665757 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:27.666352 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.667338 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.669544 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.670351 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.670551 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.670627 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.671010 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.671167 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 16:55:27.671553 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.664429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:32875 (local address 127.8.195.130:54963)
0812 16:55:27.665060 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.665078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:27.665363 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:27.665771 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:27.665775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:27.665804 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:27.666146 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.666162 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.667591 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.667596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.669384 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:27.669397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.670595 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.670601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:27.670852 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.670884 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 16:55:27.670975 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 16:55:27.671138 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":19,"thread_start_us":308,"threads_started":1}
I20260812 16:55:27.672013 4762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830e26591094f5fa146447af486a384" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b765f0bead3484392ce618472f8fc37" is_pre_election: true
I20260812 16:55:27.672667 4762 raft_consensus.cc:2454] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 0.
I20260812 16:55:27.672623 5288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830e26591094f5fa146447af486a384" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91cafd103b847b0b83029252b3e8bce" is_pre_election: true
I20260812 16:55:27.673172 5288 raft_consensus.cc:2454] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 0.
I20260812 16:55:27.673416 4892 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37, b9aa68e6863446148d24927219d964bc; no voters:
I20260812 16:55:27.673985 5407 raft_consensus.cc:2781] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:55:27.674167 5407 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:55:27.674366 5407 raft_consensus.cc:3037] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:27.676779 5407 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.677560 5407 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 election: Requested vote from peers 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
I20260812 16:55:27.678277 4762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830e26591094f5fa146447af486a384" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b765f0bead3484392ce618472f8fc37"
I20260812 16:55:27.678514 4762 raft_consensus.cc:3037] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:27.678488 5288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830e26591094f5fa146447af486a384" candidate_uuid: "b9aa68e6863446148d24927219d964bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f91cafd103b847b0b83029252b3e8bce"
I20260812 16:55:27.678745 5288 raft_consensus.cc:3037] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:27.680956 4762 raft_consensus.cc:2454] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 1.
I20260812 16:55:27.681059 5288 raft_consensus.cc:2454] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9aa68e6863446148d24927219d964bc in term 1.
I20260812 16:55:27.681516 4892 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37, b9aa68e6863446148d24927219d964bc; no voters:
I20260812 16:55:27.681862 5407 raft_consensus.cc:2781] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:27.682385 5407 raft_consensus.cc:686] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 1 LEADER]: Becoming Leader. State: Replica: b9aa68e6863446148d24927219d964bc, State: Running, Role: LEADER
I20260812 16:55:27.683033 5407 consensus_queue.cc:237] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:27.688122 4465 catalog_manager.cc:5495] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc reported cstate change: term changed from 0 to 1, leader changed from <none> to b9aa68e6863446148d24927219d964bc (127.8.195.130). New cstate: current_term: 1 leader_uuid: "b9aa68e6863446148d24927219d964bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } health_report { overall_health: UNKNOWN } } }
I20260812 16:55:27.726833 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.721264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.8.195.130:45171)
0812 16:55:27.721520 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.721527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.721548 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:55:27.721682 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.721686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.721766 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.721940 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:55:27.722333 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.723058 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.724568 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.724983 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.725124 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.725698 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:55:27.726217 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:55:27.726398 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.726458 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.726542 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:55:27.737090 5288 raft_consensus.cc:1261] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Refusing update from remote peer b9aa68e6863446148d24927219d964bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:27.737084 4762 raft_consensus.cc:1261] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Refusing update from remote peer b9aa68e6863446148d24927219d964bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:27.738245 5414 consensus_queue.cc:1035] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:27.738639 5407 consensus_queue.cc:1035] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:27.767855 4465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:27.768204 4465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:27.772532 4465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Sending DeleteTablet for 3 replicas of tablet 8830e26591094f5fa146447af486a384
I20260812 16:55:27.774115 4981 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830e26591094f5fa146447af486a384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:27 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:50634
I20260812 16:55:27.774194 5244 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830e26591094f5fa146447af486a384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:27 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:47918
I20260812 16:55:27.774191 4717 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830e26591094f5fa146447af486a384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:27 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:35554
I20260812 16:55:27.775470 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 4545
I20260812 16:55:27.786458 5443 tablet_replica.cc:330] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: stopping tablet replica
I20260812 16:55:27.787796 5443 raft_consensus.cc:2229] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:27.789409 5443 raft_consensus.cc:2258] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:27.790556 5442 tablet_replica.cc:330] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: stopping tablet replica
I20260812 16:55:27.791079 5442 raft_consensus.cc:2229] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:55:27.791813 5442 raft_consensus.cc:2258] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:27.794574 5442 ts_tablet_manager.cc:1905] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:27.801738 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.777702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.8.195.129:42387)
0812 16:55:27.777943 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.777952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.777977 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:55:27.778057 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.778066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.778173 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.778299 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:27.781296 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.782980 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.786647 (+ 3667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.787664 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.787820 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.792337 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:27.792358 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:27.792378 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:27.792427 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:27.796456 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.798243 (+ 1787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.798254 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.798257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.798448 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:27.798943 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:27.798949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:27.798953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:27.799478 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:27.799798 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:27.801051 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0812 16:55:27.801350 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:55:27.802728 4844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:27.802865 5442 ts_tablet_manager.cc:1918] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:27.803071 5442 log.cc:1196] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/8830e26591094f5fa146447af486a384
I20260812 16:55:27.803709 5442 ts_tablet_manager.cc:1939] T 8830e26591094f5fa146447af486a384 P b9aa68e6863446148d24927219d964bc: Deleting consensus metadata
I20260812 16:55:27.805826 4443 catalog_manager.cc:4841] TS b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): tablet 8830e26591094f5fa146447af486a384 (table test-table [id=588ab6cefa1046499119161451a8f697]) successfully deleted
I20260812 16:55:27.813271 5443 ts_tablet_manager.cc:1905] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:27.820358 5443 ts_tablet_manager.cc:1918] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:27.820568 5443 log.cc:1196] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/8830e26591094f5fa146447af486a384
I20260812 16:55:27.821022 5443 ts_tablet_manager.cc:1939] T 8830e26591094f5fa146447af486a384 P f91cafd103b847b0b83029252b3e8bce: Deleting consensus metadata
I20260812 16:55:27.822755 4440 catalog_manager.cc:4841] TS f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): tablet 8830e26591094f5fa146447af486a384 (table test-table [id=588ab6cefa1046499119161451a8f697]) successfully deleted
I20260812 16:55:27.950827 5445 tablet_replica.cc:330] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: stopping tablet replica
I20260812 16:55:27.951191 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 4545
I20260812 16:55:27.987411 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 4865
W20260812 16:55:27.987452 4440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:42387 (error 108)
I20260812 16:55:27.991094 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.990104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35570)
0812 16:55:27.990713 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:27.990871 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":357,"threads_started":1}
W20260812 16:55:27.991318 4440 catalog_manager.cc:4584] TS 5b765f0bead3484392ce618472f8fc37 (127.8.195.129:42387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8830e26591094f5fa146447af486a384: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
I20260812 16:55:28.003593 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:27.988468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.8.195.130:45171)
0812 16:55:27.988736 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:27.988746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:27.989008 (+ 262us) server_negotiation.cc:407] Connection header received
0812 16:55:27.989115 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:27.989120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:27.989248 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:27.989362 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:27.991195 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.992167 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.993575 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:27.994050 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:27.994205 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:27.997046 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:27.997058 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:27.997073 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:27.997107 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:28.000519 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:28.001326 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:28.001334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:28.001336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:28.001494 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:28.001935 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:28.001945 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:28.001948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:28.002322 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:28.002703 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:28.002979 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 16:55:28.003209 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:55:28.004416 5101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:28.045117 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.044551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35578)
0812 16:55:28.044799 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.044932 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 16:55:28.046392 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.045949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35580)
0812 16:55:28.046119 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.046213 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 16:55:28.107057 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.106465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35584)
0812 16:55:28.106742 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.106862 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 16:55:28.108309 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.107878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35592)
0812 16:55:28.108048 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.108133 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 16:55:28.123632 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 4865
I20260812 16:55:28.160107 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 5116
I20260812 16:55:28.174671 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.160969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.8.195.131:32875)
0812 16:55:28.161210 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:28.161242 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:28.161268 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:55:28.161439 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:28.161444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:28.161515 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:28.161638 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:28.163076 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:28.163867 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:28.165395 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:28.165838 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:28.165945 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:28.168929 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:28.168944 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:28.168961 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:28.169002 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:28.171898 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:28.172736 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:28.172744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:28.172746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:28.172902 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:28.173288 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:28.173292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:28.173294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:28.173706 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:28.173989 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:28.174205 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 16:55:28.174389 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:55:28.175410 5362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:28.189611 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.188865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35602)
0812 16:55:28.189143 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.189428 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260812 16:55:28.190898 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.190424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35618)
0812 16:55:28.190642 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.190732 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 16:55:28.267696 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 5116
I20260812 16:55:28.290946 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 4409
I20260812 16:55:28.292325 4531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:28.408917 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:28.408343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:42387 (local address 127.0.0.1:35630)
0812 16:55:28.408587 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:28.408700 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:42387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 16:55:28.408882 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 4409
I20260812 16:55:28.442052 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--webserver_interface=127.8.195.190
--webserver_port=36061
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:28.808197 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:28.808518 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:28.808614 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:28.816023 5461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:28.816146 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:28.816195 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:28.816253 5461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:28.816300 5461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:28.826571 5461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=36061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:28.827903 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:28.829614 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:28.844597 5470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:28.844676 5473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:28.844650 5471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:28.845410 5461 server_base.cc:1034] running on GCE node
I20260812 16:55:28.846558 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:28.848047 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:28.849336 5461 hybrid_clock.cc:648] HybridClock initialized: now 1786553728849290 us; error 59 us; skew 500 ppm
I20260812 16:55:28.849747 5461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:28 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553728, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:28.853024 5461 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:28.854667 5461 webserver.cc:460] Webserver started at http://127.8.195.190:36061/ using document root <none> and password file <none>
I20260812 16:55:28.855340 5461 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:28.855448 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:28.861275 5461 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 16:55:28.866758 5501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:28.868693 5461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:55:28.869037 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0"
format_stamp: "Formatted at 2026-08-12 16:55:22 on dist-test-slave-z8x9"
server_key: "6a3733fa8770d8e0fe351722766acf03"
server_key_iv: "13cd29a847ef7e38e333153e7dc2e345"
server_key_version: "encryptionkey@0"
I20260812 16:55:28.870385 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:28.896703 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:28.901098 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:28.901472 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:28.921277 5461 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34793
I20260812 16:55:28.921283 5598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34793 every 8 connection(s)
I20260812 16:55:28.922941 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:28.923934 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 5461
I20260812 16:55:28.924834 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:42387
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=40837
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:28.930626 5601 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:55:28.934417 5601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap starting.
I20260812 16:55:28.955591 5601 log.cc:824] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:28.973416 5601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:55:28.974151 5601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0: Bootstrap complete.
I20260812 16:55:28.984113 5601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:28.985302 5601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Initialized, Role: FOLLOWER
I20260812 16:55:28.986065 5601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:28.986347 5601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:28.986495 5601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:28.986680 5601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:55:28.990862 5601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:28.991406 5601 leader_election.cc:302] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c9c2b8ac3a4ee285933d5dc384d4b0; no voters:
I20260812 16:55:28.992146 5601 leader_election.cc:288] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:55:28.992302 5606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:55:28.993570 5606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [term 3 LEADER]: Becoming Leader. State: Replica: 74c9c2b8ac3a4ee285933d5dc384d4b0, State: Running, Role: LEADER
I20260812 16:55:28.994210 5606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } }
I20260812 16:55:28.994750 5601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:28.997087 5611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c9c2b8ac3a4ee285933d5dc384d4b0. Latest consensus state: current_term: 3 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:28.997509 5608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c9c2b8ac3a4ee285933d5dc384d4b0" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34793 } } }
I20260812 16:55:28.998257 5611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:28.998270 5608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c9c2b8ac3a4ee285933d5dc384d4b0 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:29.010617 5622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:29.020077 5622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:29.021735 5622 catalog_manager.cc:1259] Loaded cluster ID: 50a47809a1c748a38f9dd1ca24551fa0
I20260812 16:55:29.021860 5622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 16:55:29.024472 5622 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe738df39ad google::LogMessage::Fail() at ??:0
@ 0x7fe738df688a google::LogMessage::SendToLog() at ??:0
@ 0x7fe738df34de google::LogMessage::Flush() at ??:0
@ 0x7fe738df4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe74e82a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe74e827976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe74e8af610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
I20260812 16:55:29.063544 5658 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fe74e8af439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe73ba3f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe73ba4291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe73ba42799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe73ba1deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe73ecf76db start_thread at ??:0
@ 0x7fe737c8d61f clone at ??:0
W20260812 16:55:29.359519 5603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:29.359854 5603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:29.360057 5603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:29.367576 5603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:29.367709 5603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:29.367934 5603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:29.378242 5603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:42387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=40837
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:29.379743 5603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:29.381232 5603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:29.396162 5689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:29.396248 5686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:29.396224 5692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:29.397334 5603 server_base.cc:1034] running on GCE node
I20260812 16:55:29.397981 5603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:29.399026 5603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:29.400332 5603 hybrid_clock.cc:648] HybridClock initialized: now 1786553729400277 us; error 69 us; skew 500 ppm
I20260812 16:55:29.400717 5603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:29 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:29.404366 5603 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:29.406185 5603 webserver.cc:460] Webserver started at http://127.8.195.129:40837/ using document root <none> and password file <none>
I20260812 16:55:29.407012 5603 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:29.407188 5603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:29.413230 5603 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:55:29.416541 5708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:29.418345 5603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 16:55:29.418584 5603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "5b765f0bead3484392ce618472f8fc37"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "013356561a5882ec73ed557de3c102d1"
server_key_iv: "e1093e04f38ecb8dc5bfd33ef4a3a6bb"
server_key_version: "encryptionkey@0"
I20260812 16:55:29.419731 5603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:29.437249 5603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:29.441725 5603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:29.442039 5603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:29.443105 5603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:29.445138 5722 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 16:55:29.451334 5603 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 16:55:29.451512 5603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260812 16:55:29.451666 5603 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 16:55:29.455165 5603 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 16:55:29.455317 5603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260812 16:55:29.455713 5722 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Bootstrap starting.
I20260812 16:55:29.484140 5722 log.cc:824] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:29.502991 5722 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:55:29.503858 5722 tablet_bootstrap.cc:492] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Bootstrap complete.
I20260812 16:55:29.504616 5603 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:42387
I20260812 16:55:29.504673 5937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:42387 every 8 connection(s)
I20260812 16:55:29.505940 5722 ts_tablet_manager.cc:1397] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Time spent bootstrapping tablet: real 0.051s user 0.038s sys 0.010s
I20260812 16:55:29.506453 5603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:29.509775 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 5603
I20260812 16:55:29.510669 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:45171
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=43679
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:55:29.514115 5722 raft_consensus.cc:348] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:29.515108 5722 raft_consensus.cc:729] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b765f0bead3484392ce618472f8fc37, State: Initialized, Role: FOLLOWER
I20260812 16:55:29.515905 5722 consensus_queue.cc:260] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:29.517822 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:29.510040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:55841)
0812 16:55:29.515997 (+ 5957us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:29.516409 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5584,"thread_start_us":5169,"threads_started":1}
I20260812 16:55:29.517845 5722 ts_tablet_manager.cc:1428] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260812 16:55:29.518603 5939 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:34793 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:34793: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
I20260812 16:55:29.519579 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:29.519245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:49009)
0812 16:55:29.519396 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:29.519470 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 16:55:29.520601 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:29.520201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:54057)
0812 16:55:29.520390 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:29.520482 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 16:55:29.520855 5939 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:55:29.864925 5955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:29.865322 5955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:29.865530 5955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:29.874425 5955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:29.874552 5955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:29.874758 5955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:29.885043 5955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:45171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=43679
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:29.886538 5955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:29.888036 5955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:29.902446 5968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:29.902496 5965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:29.902549 5964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:29.906932 5955 server_base.cc:1034] running on GCE node
I20260812 16:55:29.907698 5955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:29.908787 5955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:29.910051 5955 hybrid_clock.cc:648] HybridClock initialized: now 1786553729909994 us; error 52 us; skew 500 ppm
I20260812 16:55:29.910427 5955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:29 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:29.913918 5955 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:29.915675 5955 webserver.cc:460] Webserver started at http://127.8.195.130:43679/ using document root <none> and password file <none>
I20260812 16:55:29.916488 5955 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:29.916651 5955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:29.922626 5955 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 16:55:29.925980 5987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:29.927419 5955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:55:29.927627 5955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "b9aa68e6863446148d24927219d964bc"
format_stamp: "Formatted at 2026-08-12 16:55:23 on dist-test-slave-z8x9"
server_key: "64e8f318d93526412b9a6cb3a956ff07"
server_key_iv: "19b144f6f93496115b43e76113946020"
server_key_version: "encryptionkey@0"
I20260812 16:55:29.928692 5955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:29.949656 5955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:29.954684 5955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:29.954998 5955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:29.956198 5955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:29.957806 5955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:29.957957 5955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:29.958101 5955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:29.958196 5955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:29.995136 5955 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:45171
I20260812 16:55:29.995231 6218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:45171 every 8 connection(s)
I20260812 16:55:29.996749 5955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:30.002221 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.000014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:36707)
0812 16:55:30.000872 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.001118 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":337,"threads_started":1}
W20260812 16:55:30.003055 6221 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:34793 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:34793: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
I20260812 16:55:30.004081 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.003674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:38485)
0812 16:55:30.003840 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.003929 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 16:55:30.004956 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.004627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:54403)
0812 16:55:30.004768 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.004847 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:55:30.005275 6221 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:55:30.005610 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 5955
I20260812 16:55:30.006287 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:32875
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:34793
--webserver_port=37791
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:30.359901 6225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:30.360235 6225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:30.360414 6225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:30.367988 6225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:30.368127 6225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:30.368345 6225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:30.378690 6225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:32875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=37791
--tserver_master_addrs=127.8.195.190:34793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:30.380152 6225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:30.381599 6225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:30.396400 6237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:30.396421 6235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:30.397125 6239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:30.397617 6225 server_base.cc:1034] running on GCE node
I20260812 16:55:30.398324 6225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:30.399448 6225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:30.400717 6225 hybrid_clock.cc:648] HybridClock initialized: now 1786553730400652 us; error 63 us; skew 500 ppm
I20260812 16:55:30.401093 6225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:30 dist-test-slave-z8x9 krb5kdc[3370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553730, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:30.408401 6225 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:30.410249 6225 webserver.cc:460] Webserver started at http://127.8.195.131:37791/ using document root <none> and password file <none>
I20260812 16:55:30.411051 6225 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:30.411216 6225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:30.417074 6225 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:55:30.420089 6247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:30.421680 6225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:30.421881 6225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f91cafd103b847b0b83029252b3e8bce"
format_stamp: "Formatted at 2026-08-12 16:55:24 on dist-test-slave-z8x9"
server_key: "de53f476c5f92ac4ba6d2de1fde4d4f6"
server_key_iv: "f7abdb234babc7d5707061be9d5791b7"
server_key_version: "encryptionkey@0"
I20260812 16:55:30.422962 6225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:30.437870 6225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:30.442605 6225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:30.442930 6225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:30.443981 6225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:30.445534 6225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:30.445679 6225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:30.445828 6225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:30.445920 6225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:30.482249 6225 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:32875
I20260812 16:55:30.482359 6473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:32875 every 8 connection(s)
I20260812 16:55:30.483920 6225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:30.493052 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 6225
I20260812 16:55:30.494911 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.487729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:54357)
0812 16:55:30.493264 (+ 5535us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.493602 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5053,"thread_start_us":4744,"threads_started":1}
W20260812 16:55:30.495813 6476 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:34793 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:34793: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
I20260812 16:55:30.496752 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.496385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:45559)
0812 16:55:30.496549 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.496628 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 16:55:30.497886 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.497483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:59693)
0812 16:55:30.497657 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.497754 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 16:55:30.498257 6476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:55:30.522364 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.521497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:43523)
0812 16:55:30.522008 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.522109 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":242,"threads_started":1}
I20260812 16:55:30.923305 6491 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:30.923708 6491 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:30.925709 6491 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9aa68e6863446148d24927219d964bc (127.8.195.130:45171), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
Aug 12 16:55:30 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
Aug 12 16:55:30 dist-test-slave-z8x9 krb5kdc[3370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:55:30.948233 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.926557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:56711 (local address 127.8.195.131:32875)
0812 16:55:30.928463 (+ 1906us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:30.928478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:30.929087 (+ 609us) server_negotiation.cc:407] Connection header received
0812 16:55:30.930905 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:30.930917 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:30.931331 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:30.931677 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:30.933053 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.934508 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.936385 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.936981 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.937199 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:30.941501 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:30.941509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:30.941522 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:30.941560 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:30.944486 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945266 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:30.945278 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:30.945284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:30.945451 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945914 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:30.945919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:30.945927 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:30.946428 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:30.946631 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:30.947443 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 16:55:30.947800 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1683,"thread_start_us":1545,"threads_started":1}
I20260812 16:55:30.948251 6494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.926265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:52827 (local address 127.8.195.130:45171)
0812 16:55:30.928191 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:30.928212 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:30.928266 (+ 54us) server_negotiation.cc:407] Connection header received
0812 16:55:30.930242 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:30.930253 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:30.930638 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:30.930908 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:30.933061 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.934468 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.936386 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.937026 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.937306 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:30.941636 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:30.941643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:30.941662 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:30.941699 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:30.944901 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945655 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:30.945666 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:30.945671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:30.945833 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:30.946248 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:30.946253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:30.946257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:30.946718 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:30.946915 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:30.947463 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 16:55:30.947841 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1643,"thread_start_us":1486,"threads_started":1}
I20260812 16:55:30.948248 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.926007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:45171 (local address 127.8.195.129:52827)
0812 16:55:30.926409 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.926455 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:30.927177 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:30.931121 (+ 3944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:30.931132 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:30.932074 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:30.932855 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:30.932877 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.934681 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.934690 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:30.936140 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:30.936152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.937302 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.937309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:30.937549 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:30.938363 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:30.938387 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:30.941384 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:30.945047 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:30.945058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:30.945509 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:30.945958 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:30.945967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:30.946126 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:30.946852 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:30.946857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:30.947011 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 16:55:30.947206 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 16:55:30.947755 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":118}
I20260812 16:55:30.948377 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:30.926019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:32875 (local address 127.8.195.129:56711)
0812 16:55:30.928967 (+ 2948us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:30.928987 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:30.929240 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:30.931627 (+ 2387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:30.931634 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:30.932210 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:30.932854 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:30.932879 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.934729 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.934733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:30.936139 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:30.936153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:30.937230 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:30.937238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:30.937551 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:30.938421 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:30.938438 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:30.941094 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:30.944641 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:30.944650 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:30.944665 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:30.945092 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:30.945574 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:30.945579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:30.945583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:30.945787 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:30.946572 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:30.946582 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:30.946889 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 16:55:30.947206 (+ 317us) client_negotiation.cc:240] Negotiation successful
0812 16:55:30.947920 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2815,"mutex_wait_us":585,"spinlock_wait_cycles":13568,"thread_start_us":2376,"threads_started":1}
W20260812 16:55:30.951584 5710 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
W20260812 16:55:30.951886 5715 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
I20260812 16:55:30.952100 5715 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37; no voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce
I20260812 16:55:30.952553 6491 raft_consensus.cc:2726] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:55:31.006726 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:31.005803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:41257)
0812 16:55:31.006318 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:31.006471 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":258,"threads_started":1}
I20260812 16:55:31.500020 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:31.498934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:51703)
0812 16:55:31.499592 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:31.499725 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":407,"threads_started":1}
I20260812 16:55:31.524106 6500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:31.523409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:51471)
0812 16:55:31.523848 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:31.523932 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260812 16:55:32.008613 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:32.007755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:36875)
0812 16:55:32.008330 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:32.008435 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":291,"threads_started":1}
I20260812 16:55:32.501814 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:32.500924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:58465)
0812 16:55:32.501493 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:32.501617 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":310,"threads_started":1}
I20260812 16:55:32.525923 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:32.525059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:46649)
0812 16:55:32.525606 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:32.525709 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":308,"threads_started":1}
I20260812 16:55:32.586462 6508 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:32.586757 6508 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:32.587607 6508 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9aa68e6863446148d24927219d964bc (127.8.195.130:45171), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
W20260812 16:55:32.588685 5710 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
W20260812 16:55:32.588851 5715 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
I20260812 16:55:32.589004 5715 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37; no voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce
I20260812 16:55:32.589298 6508 raft_consensus.cc:2726] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:55:33.010550 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:33.009740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:51409)
0812 16:55:33.010274 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:33.010372 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260812 16:55:33.503515 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:33.502784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:51973)
0812 16:55:33.503239 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:33.503332 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260812 16:55:33.527956 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:33.527071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:55815)
0812 16:55:33.527688 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:33.527783 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":370,"threads_started":1}
I20260812 16:55:34.012517 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:34.011612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:53909)
0812 16:55:34.012215 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:34.012315 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":282,"threads_started":1}
I20260812 16:55:34.505595 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:34.504601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:57727)
0812 16:55:34.505237 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:34.505352 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":319,"threads_started":1}
I20260812 16:55:34.530161 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:34.529329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:55577)
0812 16:55:34.529862 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:34.529949 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":259,"threads_started":1}
I20260812 16:55:35.014599 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:35.013757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:55193)
0812 16:55:35.014278 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:35.014392 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":311,"threads_started":1}
I20260812 16:55:35.507679 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:35.506823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:54525)
0812 16:55:35.507349 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:35.507448 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":305,"threads_started":1}
I20260812 16:55:35.531944 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:35.531166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:46029)
0812 16:55:35.531671 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:35.531767 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":262,"threads_started":1}
I20260812 16:55:35.699218 6520 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:35.699528 6520 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:35.700327 6520 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9aa68e6863446148d24927219d964bc (127.8.195.130:45171), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
W20260812 16:55:35.701404 5715 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
W20260812 16:55:35.701607 5710 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
I20260812 16:55:35.701728 5710 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37; no voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce
I20260812 16:55:35.701980 6520 raft_consensus.cc:2726] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:55:36.016494 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:36.015615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:40397)
0812 16:55:36.016192 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:36.016284 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":334,"threads_started":1}
I20260812 16:55:36.509686 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:36.508713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:60081)
0812 16:55:36.509293 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:36.509418 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":289,"threads_started":1}
I20260812 16:55:36.533874 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:36.532935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:36817)
0812 16:55:36.533579 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:36.533664 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":394,"threads_started":1}
I20260812 16:55:37.018285 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:37.017502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:60367)
0812 16:55:37.018007 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:37.018091 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":223,"threads_started":1}
I20260812 16:55:37.511843 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:37.510728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:58843)
0812 16:55:37.511396 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:37.511543 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":304,"threads_started":1}
I20260812 16:55:37.535745 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:37.534891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:58193)
0812 16:55:37.535485 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:37.535579 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":340,"threads_started":1}
I20260812 16:55:38.020128 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:38.019368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:56671)
0812 16:55:38.019837 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:38.019926 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":259,"threads_started":1}
I20260812 16:55:38.513729 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:38.512805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:50673)
0812 16:55:38.513385 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:38.513503 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":271,"threads_started":1}
I20260812 16:55:38.537710 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:38.536751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:51671)
0812 16:55:38.537381 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:38.537485 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":273,"threads_started":1}
I20260812 16:55:38.629473 6541 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:38.629720 6541 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:38.630477 6541 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9aa68e6863446148d24927219d964bc (127.8.195.130:45171), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
W20260812 16:55:38.631456 5710 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
W20260812 16:55:38.631635 5715 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
I20260812 16:55:38.631829 5715 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37; no voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce
I20260812 16:55:38.632095 6541 raft_consensus.cc:2726] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:55:39.022262 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:39.021446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:48059)
0812 16:55:39.021963 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:39.022065 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":252,"threads_started":1}
I20260812 16:55:39.515329 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:39.514599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:43543)
0812 16:55:39.515077 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:39.515155 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260812 16:55:39.539636 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:39.538906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:40861)
0812 16:55:39.539386 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:39.539469 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":245,"threads_started":1}
I20260812 16:55:40.024269 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:40.023386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:40659)
0812 16:55:40.023928 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:40.024036 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":308,"threads_started":1}
I20260812 16:55:40.517020 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:40.516342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:60723)
0812 16:55:40.516761 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:40.516839 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260812 16:55:40.541424 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:40.540611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:48117)
0812 16:55:40.541061 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:40.541156 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260812 16:55:41.026262 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:41.025482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:49545)
0812 16:55:41.025992 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:41.026085 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":254,"threads_started":1}
I20260812 16:55:41.519073 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:41.518179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:33341)
0812 16:55:41.518766 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:41.518852 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":316,"threads_started":1}
I20260812 16:55:41.543128 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:41.542400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:56547)
0812 16:55:41.542857 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:41.542943 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":270,"threads_started":1}
I20260812 16:55:42.028077 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:42.027252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:45959)
0812 16:55:42.027777 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:42.027887 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260812 16:55:42.221335 6563 raft_consensus.cc:482] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:42.221555 6563 raft_consensus.cc:504] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b765f0bead3484392ce618472f8fc37" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 42387 } } peers { permanent_uuid: "b9aa68e6863446148d24927219d964bc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 45171 } } peers { permanent_uuid: "f91cafd103b847b0b83029252b3e8bce" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 32875 } }
I20260812 16:55:42.222199 6563 leader_election.cc:288] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9aa68e6863446148d24927219d964bc (127.8.195.130:45171), f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875)
W20260812 16:55:42.223291 5715 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9aa68e6863446148d24927219d964bc (127.8.195.130:45171): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
W20260812 16:55:42.223498 5710 leader_election.cc:341] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f91cafd103b847b0b83029252b3e8bce (127.8.195.131:32875): Not found: Tablet not found: 8830e26591094f5fa146447af486a384
I20260812 16:55:42.223635 5710 leader_election.cc:302] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b765f0bead3484392ce618472f8fc37; no voters: b9aa68e6863446148d24927219d964bc, f91cafd103b847b0b83029252b3e8bce
I20260812 16:55:42.223851 6563 raft_consensus.cc:2726] T 8830e26591094f5fa146447af486a384 P 5b765f0bead3484392ce618472f8fc37 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 16:55:42.520972 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:42.520191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:38833)
0812 16:55:42.520689 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:42.520776 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":250,"threads_started":1}
I20260812 16:55:42.545149 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:42.544175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:46839)
0812 16:55:42.544785 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:42.544910 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":296,"threads_started":1}
I20260812 16:55:43.029960 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:43.029036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:43191)
0812 16:55:43.029674 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:43.029773 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":318,"threads_started":1}
I20260812 16:55:43.522964 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:43.522018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:53699)
0812 16:55:43.522681 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:43.522797 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":411,"threads_started":1}
I20260812 16:55:43.547035 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:43.546228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:44769)
0812 16:55:43.546733 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:43.546827 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":251,"threads_started":1}
I20260812 16:55:44.032095 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:44.031098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:53535)
0812 16:55:44.031643 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:44.031798 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":259,"threads_started":1}
I20260812 16:55:44.524952 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:44.523967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:43183)
0812 16:55:44.524577 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:44.524694 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":387,"threads_started":1}
I20260812 16:55:44.549376 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:44.548441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:39843)
0812 16:55:44.548953 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:44.549059 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":260,"threads_started":1}
I20260812 16:55:45.034255 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.033411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.130:44971)
0812 16:55:45.033952 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:45.034087 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":293,"threads_started":1}
I20260812 16:55:45.501566 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 5603
I20260812 16:55:45.518445 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.502838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.8.195.129:42387)
0812 16:55:45.503459 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:45.503471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:45.503504 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:55:45.503609 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:45.503623 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:45.503961 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:45.504110 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:45.505591 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.506744 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.508573 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.509231 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.509352 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:45.512412 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:45.512425 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:45.512447 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:45.512486 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:45.515789 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.516469 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.516476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.516478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.516623 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.516987 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.516990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.516992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.517466 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:45.517703 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:45.517971 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 16:55:45.518178 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":261,"threads_started":1}
I20260812 16:55:45.519727 5933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:45.527117 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.526241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.131:43965)
0812 16:55:45.526741 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:45.526919 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260812 16:55:45.612288 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.611167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34793 (local address 127.8.195.129:58661)
0812 16:55:45.611754 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:45.611936 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":259,"threads_started":1}
I20260812 16:55:45.612402 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 5603
I20260812 16:55:45.646435 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 5955
I20260812 16:55:45.661725 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.647325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.8.195.130:45171)
0812 16:55:45.647864 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:45.647869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:45.647886 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:55:45.647944 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:45.647948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:45.648010 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:45.648111 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:45.649668 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.650545 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.651834 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.652292 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.652397 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:45.655603 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:45.655609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:45.655612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:45.655654 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:45.659058 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.659863 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.659869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.659878 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.660015 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.660369 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.660372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.660374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.660672 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:45.660884 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:45.661160 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 16:55:45.661427 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":302,"threads_started":1}
I20260812 16:55:45.662578 6207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:45.747969 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 5955
I20260812 16:55:45.778177 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 6225
I20260812 16:55:45.792958 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:45.779152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.8.195.131:32875)
0812 16:55:45.779755 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:45.779760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:45.779777 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:55:45.779833 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:45.779836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:45.779907 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:45.780033 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:45.781619 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.782485 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.783688 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:45.784132 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:45.784273 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:45.787149 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:45.787155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:45.787158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:45.787191 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:45.790088 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.790825 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.790839 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.790842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.790989 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:45.791402 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:45.791408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:45.791411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:45.791694 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:45.792010 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:45.792304 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 16:55:45.792547 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":312,"threads_started":1}
I20260812 16:55:45.793810 6463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:45.879606 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 6225
2026-08-12T16:55:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25668 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:55:46 dist-test-slave-z8x9 krb5kdc[6615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:55:46 dist-test-slave-z8x9 krb5kdc[6615](info): set up 2 sockets
Aug 12 16:55:46 dist-test-slave-z8x9 krb5kdc[6615](info): commencing operation
krb5kdc: starting...
W20260812 16:55:47.310730 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.376s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:55:47 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:55:47.632375 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:47.990270 6644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:47.990619 6644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:47.990790 6644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:47.998324 6644 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:55:47.998452 6644 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:55:47.998531 6644 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:55:47.998620 6644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:55:47.998731 6644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:47.998871 6644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:47.998945 6644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:55:47.999033 6644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:55:48.009547 6644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:48.011026 6644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:48.012486 6644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:48.026448 6667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:48.026563 6654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:48.026533 6662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:48.027753 6644 server_base.cc:1034] running on GCE node
I20260812 16:55:48.028630 6644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:48.030246 6644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:48.031482 6644 hybrid_clock.cc:648] HybridClock initialized: now 1786553748031428 us; error 37 us; skew 500 ppm
I20260812 16:55:48.031867 6644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:48 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:48.035377 6644 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:55:48.036978 6644 webserver.cc:460] Webserver started at http://127.8.195.190:38391/ using document root <none> and password file <none>
I20260812 16:55:48.037766 6644 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:48.037874 6644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:48.038160 6644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:48.041357 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "6b1fe827500045ada311fd9731923601"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "9ecb4a9ed998a3e1015b2958745869a5"
server_key_iv: "5b21d0839e2fbd26fd14159cbc32fd4c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.042186 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "6b1fe827500045ada311fd9731923601"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "9ecb4a9ed998a3e1015b2958745869a5"
server_key_iv: "5b21d0839e2fbd26fd14159cbc32fd4c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.047746 6644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:55:48.051568 6688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:48.053383 6644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:55:48.053575 6644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "6b1fe827500045ada311fd9731923601"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "9ecb4a9ed998a3e1015b2958745869a5"
server_key_iv: "5b21d0839e2fbd26fd14159cbc32fd4c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.053821 6644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:48.075870 6644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:48.080453 6644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:48.080772 6644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:48.101641 6644 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37423
I20260812 16:55:48.101655 6790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37423 every 8 connection(s)
I20260812 16:55:48.103108 6644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:55:48.104338 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 6644
I20260812 16:55:48.104638 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:55:48.105746 8974 external_mini_cluster.cc:1428] Setting key b4e160b4f3b289cb2b7103725e72438f
I20260812 16:55:48.108702 6792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:55:48 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:48.122849 6792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap starting.
I20260812 16:55:48.126544 6792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:48.127995 6792 log.cc:824] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:48.131129 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:48.108009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.8.195.190:37423)
0812 16:55:48.108897 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:48.108910 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:48.108951 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:55:48.110639 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:48.110661 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:48.111042 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:48.111410 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:48.113524 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.114955 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.116843 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.117681 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.117831 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:48.123074 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:48.123090 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:48.123108 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:48.123153 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:48.126627 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:48.127610 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:48.127618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:48.127624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:48.127815 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:48.128295 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:48.128299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:48.128301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:48.128877 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:48.129258 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:48.129731 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 16:55:48.130150 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":420,"threads_started":1}
I20260812 16:55:48.131172 6792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: No bootstrap required, opened a new log
I20260812 16:55:48.139394 6792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:55:48.139822 6792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:48.139935 6792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1fe827500045ada311fd9731923601, State: Initialized, Role: FOLLOWER
I20260812 16:55:48.140614 6792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:55:48.140885 6792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:55:48.140997 6792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:55:48.141126 6792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:48.143368 6792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:55:48.143812 6792 leader_election.cc:302] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b1fe827500045ada311fd9731923601; no voters:
I20260812 16:55:48.144493 6792 leader_election.cc:288] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:55:48.144655 6806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:48.145542 6806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 LEADER]: Becoming Leader. State: Replica: 6b1fe827500045ada311fd9731923601, State: Running, Role: LEADER
I20260812 16:55:48.146067 6806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:55:48.146689 6792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:55:48.148983 6807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:55:48.149837 6807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:48.148985 6809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b1fe827500045ada311fd9731923601. Latest consensus state: current_term: 1 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:55:48.151408 6809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:55:48.154235 6816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:55:48.160511 6816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:55:48.170382 6816 catalog_manager.cc:1347] Generated new cluster ID: dcb0331b8a574e959aa7f53a7b77323d
I20260812 16:55:48.170533 6816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:55:48.180900 6816 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:55:48.182435 6816 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:55:48.192788 6816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Generated new TSK 0
I20260812 16:55:48.193811 6816 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:55:48.273104 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:48.637770 6845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:48.638186 6845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:48.638377 6845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:48.650208 6845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:48.650328 6845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:48.650451 6845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:55:48.661151 6845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:48.662616 6845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:48.663991 6845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:48.679461 6856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:48.679427 6854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:48.679523 6865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:48.680372 6845 server_base.cc:1034] running on GCE node
I20260812 16:55:48.681082 6845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:48.682135 6845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:48.683411 6845 hybrid_clock.cc:648] HybridClock initialized: now 1786553748683356 us; error 77 us; skew 500 ppm
I20260812 16:55:48.683759 6845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:48 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:48.687376 6845 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:55:48.689093 6845 webserver.cc:460] Webserver started at http://127.8.195.129:43181/ using document root <none> and password file <none>
I20260812 16:55:48.689874 6845 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:48.690032 6845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:48.690379 6845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:48.693673 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "293b377314ea40293a09d71f7560e197"
server_key_iv: "dce4bf05adede73239fd949b8bac069c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.694526 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "293b377314ea40293a09d71f7560e197"
server_key_iv: "dce4bf05adede73239fd949b8bac069c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.700232 6845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:55:48.704125 6878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:48.705704 6845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:55:48.705914 6845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "293b377314ea40293a09d71f7560e197"
server_key_iv: "dce4bf05adede73239fd949b8bac069c"
server_key_version: "encryptionkey@0"
I20260812 16:55:48.706179 6845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:48.730235 6845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:48.735137 6845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:48.735491 6845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:48.736584 6845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:48.738044 6845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:48.738222 6845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:48.738379 6845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:48.738456 6845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:48.774677 6845 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:37517
I20260812 16:55:48.774775 7093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:37517 every 8 connection(s)
I20260812 16:55:48.776041 6845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:55:48.777758 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 6845
I20260812 16:55:48.777987 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:55:48.778893 8974 external_mini_cluster.cc:1428] Setting key 03111d593ec06a031023fd355f4acbbd
Aug 12 16:55:48 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:48.798485 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:48.779022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:59187 (local address 127.8.195.190:37423)
0812 16:55:48.779537 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:48.779544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:48.780583 (+ 1039us) server_negotiation.cc:407] Connection header received
0812 16:55:48.782736 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:48.782742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:48.782832 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:48.782952 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:48.784498 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.785352 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.786942 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.787359 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.787504 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:48.791719 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:48.791726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:48.791730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:48.791767 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:48.795124 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:48.795942 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:48.795948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:48.795953 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:48.796086 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:48.796555 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:48.796561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:48.796564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:48.796848 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:48.797013 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:48.797839 (+ 826us) server_negotiation.cc:299] Negotiation successful
0812 16:55:48.798090 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":252,"threads_started":1}
I20260812 16:55:48.799523 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:48.779556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:59187)
0812 16:55:48.780387 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:48.780437 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:48.782436 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:48.783206 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:48.783214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:48.783748 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:48.784324 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:48.784342 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.785530 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.785534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:48.786779 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:48.786792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:48.787621 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:48.787629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:48.787887 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:48.788732 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:48.788760 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:48.791437 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:48.795303 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:48.795314 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:48.795339 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:48.795805 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:48.796204 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:48.796209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:48.796212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:48.796433 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:48.796986 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:48.796994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:48.797446 (+ 452us) client_negotiation.cc:769] Sending connection context
0812 16:55:48.797746 (+ 300us) client_negotiation.cc:240] Negotiation successful
0812 16:55:48.798321 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":322,"threads_started":1}
I20260812 16:55:48.801301 7094 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:55:48.801666 7094 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:48.802647 7094 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:55:48.805339 6717 ts_manager.cc:194] Registered new tserver with Master: 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517)
I20260812 16:55:48.807660 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:59187
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:55:48.853286 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:49.200399 7109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:49.200712 7109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:49.200812 7109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:49.208295 7109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:49.208424 7109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:49.208544 7109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:55:49.218879 7109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:49.220182 7109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:49.221596 7109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:49.236976 7124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:49.237049 7125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:49.237021 7127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:49.237845 7109 server_base.cc:1034] running on GCE node
I20260812 16:55:49.238526 7109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:49.239568 7109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:49.240808 7109 hybrid_clock.cc:648] HybridClock initialized: now 1786553749240755 us; error 34 us; skew 500 ppm
I20260812 16:55:49.241222 7109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:49 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:49.244694 7109 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:55:49.246235 7109 webserver.cc:460] Webserver started at http://127.8.195.130:33753/ using document root <none> and password file <none>
I20260812 16:55:49.246973 7109 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:49.247133 7109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:49.247527 7109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:49.250560 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "572c3d3778284c0ea3251f5e58b1eac8"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "d9274b48d4b4aa2ed66ac35679d77bea"
server_key_iv: "7a5eaa363b4100090a9e39a757787f19"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.251332 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "572c3d3778284c0ea3251f5e58b1eac8"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "d9274b48d4b4aa2ed66ac35679d77bea"
server_key_iv: "7a5eaa363b4100090a9e39a757787f19"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.257761 7109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 16:55:49.261888 7146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.263554 7109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:55:49.263705 7109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "572c3d3778284c0ea3251f5e58b1eac8"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "d9274b48d4b4aa2ed66ac35679d77bea"
server_key_iv: "7a5eaa363b4100090a9e39a757787f19"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.263885 7109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:49.279397 7109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:49.283998 7109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:49.284269 7109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:49.285284 7109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:49.286595 7109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:49.286672 7109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.286755 7109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:49.286809 7109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.321102 7109 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40923
I20260812 16:55:49.321156 7373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40923 every 8 connection(s)
I20260812 16:55:49.322515 7109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:55:49.324873 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 7109
I20260812 16:55:49.325093 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:55:49.325789 8974 external_mini_cluster.cc:1428] Setting key f30d6162fe9e8004fc40e97c53fd51c0
Aug 12 16:55:49 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:49.346479 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.325267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:44375 (local address 127.8.195.190:37423)
0812 16:55:49.325783 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.325790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.327178 (+ 1388us) server_negotiation.cc:407] Connection header received
0812 16:55:49.329692 (+ 2514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.329696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.329758 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.329853 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:49.331987 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.332748 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.335241 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.335674 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.335783 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.339943 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:49.339953 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:49.339961 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:49.340008 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:49.343099 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.343907 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.343912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.343914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.344054 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.344547 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.344551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.344553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.344847 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:49.345013 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.345824 (+ 811us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.346071 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":243,"threads_started":1}
I20260812 16:55:49.347601 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.325900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:44375)
0812 16:55:49.327003 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:49.327063 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:49.329397 (+ 2334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:49.330122 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:49.330137 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:49.330959 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:49.331780 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.331810 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.332905 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.332913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.335086 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.335104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.335907 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.335917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.336264 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.337136 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:49.337161 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:49.339675 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:49.343261 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:49.343271 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:49.343286 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:49.343737 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:49.344180 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:49.344185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:49.344191 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:49.344406 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:49.344986 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:49.344995 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:49.345445 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 16:55:49.345719 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 16:55:49.346346 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":318,"threads_started":1}
I20260812 16:55:49.349258 7374 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:55:49.349599 7374 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:49.350366 7374 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:55:49.352029 6717 ts_manager.cc:194] Registered new tserver with Master: 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923)
I20260812 16:55:49.353371 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:44375
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:55:49.400444 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:55:49.750834 7385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:55:49.751144 7385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:55:49.751355 7385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:55:49.759083 7385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:55:49.759243 7385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:55:49.759435 7385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:55:49.770007 7385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:55:49.771432 7385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:55:49.772878 7385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:55:49.787420 7396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:49.787483 7400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:55:49.787456 7405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:55:49.788331 7385 server_base.cc:1034] running on GCE node
I20260812 16:55:49.789104 7385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:55:49.790288 7385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:55:49.791498 7385 hybrid_clock.cc:648] HybridClock initialized: now 1786553749791431 us; error 62 us; skew 500 ppm
I20260812 16:55:49.791836 7385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:55:49 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:55:49.795333 7385 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:55:49.796981 7385 webserver.cc:460] Webserver started at http://127.8.195.131:45865/ using document root <none> and password file <none>
I20260812 16:55:49.797785 7385 fs_manager.cc:359] Metadata directory not provided
I20260812 16:55:49.797938 7385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:55:49.798281 7385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:55:49.801362 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "f2562853f325488b9035a1f3caca5a53"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "fe6786ac79dd9eef8804cb1187e2199a"
server_key_iv: "967bdfcd413d56563f9a3fe89b7ba53d"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.802136 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "f2562853f325488b9035a1f3caca5a53"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "fe6786ac79dd9eef8804cb1187e2199a"
server_key_iv: "967bdfcd413d56563f9a3fe89b7ba53d"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.807282 7385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:55:49.810791 7416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.811898 7094 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:55:49.812391 7385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:55:49.812585 7385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f2562853f325488b9035a1f3caca5a53"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "fe6786ac79dd9eef8804cb1187e2199a"
server_key_iv: "967bdfcd413d56563f9a3fe89b7ba53d"
server_key_version: "encryptionkey@0"
I20260812 16:55:49.812825 7385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:55:49.839773 7385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:55:49.844516 7385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:55:49.844794 7385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:55:49.845813 7385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:55:49.847281 7385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:55:49.847362 7385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.847451 7385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:55:49.847496 7385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:55:49.883930 7385 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:41759
I20260812 16:55:49.883985 7636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:41759 every 8 connection(s)
I20260812 16:55:49.885315 7385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:55:49.893826 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 7385
I20260812 16:55:49.894024 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:55:49.894598 8974 external_mini_cluster.cc:1428] Setting key d44dac8653f7b4c5a22ee13badc833b0
Aug 12 16:55:49 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:55:49.906061 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.888104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:35775 (local address 127.8.195.190:37423)
0812 16:55:49.888866 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.888872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.889607 (+ 735us) server_negotiation.cc:407] Connection header received
0812 16:55:49.891516 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.891520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.891585 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.891688 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:49.893165 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.894421 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.896318 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.896767 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.896921 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.900509 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:49.900513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:49.900517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:49.900550 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:49.903304 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.903983 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.903986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.903989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.904109 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.904508 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.904511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.904513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.904775 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:49.904893 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.905629 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.905798 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":329,"threads_started":1}
I20260812 16:55:49.907058 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.888564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:35775)
0812 16:55:49.889438 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:49.889491 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:49.891223 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:49.891893 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:49.891901 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:49.892430 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:55:49.892999 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.893015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.894583 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.894587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.896138 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.896150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.897016 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.897021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.897335 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.898098 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:55:49.898121 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:55:49.900324 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:55:49.903433 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:49.903440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:49.903452 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:49.903872 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:49.904203 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:55:49.904206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:55:49.904208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:55:49.904413 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:55:49.904889 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:55:49.904895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:55:49.905296 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 16:55:49.905540 (+ 244us) client_negotiation.cc:240] Negotiation successful
0812 16:55:49.905928 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":359,"threads_started":1}
I20260812 16:55:49.908532 7637 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:55:49.908918 7637 heartbeater.cc:464] Registering TS with master...
I20260812 16:55:49.909900 7637 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:55:49.911577 6717 ts_manager.cc:194] Registered new tserver with Master: f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759)
I20260812 16:55:49.912367 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:35775
I20260812 16:55:49.923312 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:55:49.941722 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.928719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.8.195.190:37423)
0812 16:55:49.928916 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.928921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.929374 (+ 453us) server_negotiation.cc:407] Connection header received
0812 16:55:49.929543 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.929547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.929615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.929707 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:49.931172 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.931937 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.933402 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.933821 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.933919 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.936388 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:49.936398 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:49.936402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:49.936440 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:49.939402 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.940035 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.940039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.940041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.940157 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:49.940486 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:49.940489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:49.940491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:49.940807 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:49.940985 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.941323 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.941492 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:55:49.946081 6717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:55:49.954545 6717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:55:49.979452 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.966978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40923 (local address 127.0.0.1:57068)
0812 16:55:49.968071 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:49.968122 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:49.968459 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:49.970656 (+ 2197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:49.970667 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:49.970709 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:49.971225 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.971243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.973684 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.973689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.976547 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.976564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.978175 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.978182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.978376 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.978428 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:55:49.978593 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 16:55:49.978945 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":97,"thread_start_us":472,"threads_started":1}
I20260812 16:55:49.979506 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.967231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.8.195.130:40923)
0812 16:55:49.969857 (+ 2626us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.969870 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.969908 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:55:49.970039 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.970052 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.970349 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.971482 (+ 1133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:49.971841 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.973521 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.976785 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.977960 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.978119 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.978338 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.978979 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.979160 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2458,"thread_start_us":2337,"threads_started":1}
I20260812 16:55:49.980698 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.967750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:41759 (local address 127.0.0.1:47154)
0812 16:55:49.969137 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:49.969160 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:49.969329 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:49.972388 (+ 3059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:49.972393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:49.972421 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:49.972836 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.972850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.974653 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.974661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.976619 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.976637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.978603 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.978610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.979963 (+ 1353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.980003 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 16:55:49.980085 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 16:55:49.980252 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":108,"thread_start_us":3915,"threads_started":1}
I20260812 16:55:49.981532 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.968251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:51762)
0812 16:55:49.971938 (+ 3687us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:49.971961 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:49.972082 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:49.973269 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:49.973276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:49.973301 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:49.973684 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.973693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.975843 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.975849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.978072 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:49.978088 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.980548 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.980556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:49.980750 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.980906 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 16:55:49.980991 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:55:49.981138 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3550,"mutex_wait_us":2,"thread_start_us":862,"threads_started":1}
I20260812 16:55:49.981568 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.968315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.8.195.131:41759)
0812 16:55:49.970856 (+ 2541us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.970867 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.970907 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:55:49.971023 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.971033 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.971303 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.973066 (+ 1763us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:49.973389 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.974507 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.976846 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.978389 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.978518 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.978739 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.980947 (+ 2208us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.981148 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":2175,"threads_started":1}
I20260812 16:55:49.982429 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:49.968915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.8.195.129:37517)
0812 16:55:49.972360 (+ 3445us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:49.972368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:49.972395 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:55:49.972667 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:49.972676 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:49.972911 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:49.973131 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:49.974069 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.975670 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.978282 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:49.980022 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:49.980160 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:49.980377 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:49.981923 (+ 1546us) server_negotiation.cc:299] Negotiation successful
0812 16:55:49.982099 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3274,"thread_start_us":3110,"threads_started":1}
I20260812 16:55:49.984494 7247 tablet_service.cc:1467] Processing CreateTablet for tablet 68860c5952844c61a88afbcc1bcabe2c (DEFAULT_TABLE table=test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:49.984897 7501 tablet_service.cc:1467] Processing CreateTablet for tablet 68860c5952844c61a88afbcc1bcabe2c (DEFAULT_TABLE table=test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:49.985920 7247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68860c5952844c61a88afbcc1bcabe2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:49.986377 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68860c5952844c61a88afbcc1bcabe2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:49.985927 6976 tablet_service.cc:1467] Processing CreateTablet for tablet 68860c5952844c61a88afbcc1bcabe2c (DEFAULT_TABLE table=test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:55:49.987282 6976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68860c5952844c61a88afbcc1bcabe2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:55:49.997557 7674 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Bootstrap starting.
I20260812 16:55:50.001574 7677 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Bootstrap starting.
I20260812 16:55:50.001626 7674 tablet_bootstrap.cc:654] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:50.003393 7674 log.cc:824] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:50.003521 7678 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Bootstrap starting.
I20260812 16:55:50.005386 7677 tablet_bootstrap.cc:654] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:50.006711 7677 log.cc:824] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:50.006786 7678 tablet_bootstrap.cc:654] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Neither blocks nor log segments found. Creating new log.
I20260812 16:55:50.007400 7674 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: No bootstrap required, opened a new log
I20260812 16:55:50.007669 7674 ts_tablet_manager.cc:1397] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260812 16:55:50.007997 7678 log.cc:824] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Log is configured to *not* fsync() on all Append() calls
I20260812 16:55:50.010768 7677 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: No bootstrap required, opened a new log
I20260812 16:55:50.011035 7677 ts_tablet_manager.cc:1397] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 16:55:50.018860 7678 tablet_bootstrap.cc:492] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: No bootstrap required, opened a new log
I20260812 16:55:50.018574 7674 raft_consensus.cc:348] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.019068 7674 raft_consensus.cc:374] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:50.019177 7674 raft_consensus.cc:729] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572c3d3778284c0ea3251f5e58b1eac8, State: Initialized, Role: FOLLOWER
I20260812 16:55:50.019199 7678 ts_tablet_manager.cc:1397] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260812 16:55:50.019958 7674 consensus_queue.cc:260] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.021250 7374 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:55:50.022878 7677 raft_consensus.cc:348] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.023357 7677 raft_consensus.cc:374] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:50.023408 7674 ts_tablet_manager.cc:1428] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260812 16:55:50.023460 7677 raft_consensus.cc:729] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2562853f325488b9035a1f3caca5a53, State: Initialized, Role: FOLLOWER
I20260812 16:55:50.024154 7677 consensus_queue.cc:260] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.025313 7637 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:55:50.026186 7677 ts_tablet_manager.cc:1428] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260812 16:55:50.029036 7678 raft_consensus.cc:348] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.029482 7678 raft_consensus.cc:374] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:55:50.029618 7678 raft_consensus.cc:729] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe9dc22c1d041ffa92731207ba5ec49, State: Initialized, Role: FOLLOWER
I20260812 16:55:50.030227 7678 consensus_queue.cc:260] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.031873 7678 ts_tablet_manager.cc:1428] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260812 16:55:50.074687 7375 tablet.cc:2369] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:55:50.137095 7639 tablet.cc:2369] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:55:50.278854 7103 tablet.cc:2369] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:55:50.364084 7688 raft_consensus.cc:482] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:55:50.364446 7688 raft_consensus.cc:504] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.366155 7688 leader_election.cc:288] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923), 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517)
I20260812 16:55:50.373160 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.366516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40923 (local address 127.8.195.131:57105)
0812 16:55:50.366708 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:50.366730 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:50.366862 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:50.367477 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:50.367482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:50.367511 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:50.367768 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:50.367783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.369071 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.369075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:50.371135 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:50.371151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.372549 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.372553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:50.372680 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.372704 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 16:55:50.372796 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 16:55:50.372888 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":33}
I20260812 16:55:50.373270 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.366585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:57105 (local address 127.8.195.130:40923)
0812 16:55:50.366778 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.366782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.366804 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:55:50.367027 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.367033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.367114 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.367258 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:50.367913 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.368932 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.371305 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.372393 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.372564 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.372642 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.372842 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.372975 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 16:55:50.373991 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.367001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.8.195.131:39169)
0812 16:55:50.367597 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 16:55:50.367614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:55:50.367742 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:55:50.368118 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:55:50.368123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:55:50.368152 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:55:50.368447 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:50.368455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.369714 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.369719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:50.371604 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:55:50.371615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.373228 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.373234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:55:50.373441 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.373472 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:55:50.373575 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 16:55:50.373684 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":1,"thread_start_us":298,"threads_started":1}
I20260812 16:55:50.374002 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.367313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:39169 (local address 127.8.195.129:37517)
0812 16:55:50.367519 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.367524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.367700 (+ 176us) server_negotiation.cc:407] Connection header received
0812 16:55:50.367884 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.367889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.367967 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.368098 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:55:50.368593 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.369562 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.371760 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.372991 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.373144 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.373243 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.373634 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.373771 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:55:50.374476 7287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68860c5952844c61a88afbcc1bcabe2c" candidate_uuid: "f2562853f325488b9035a1f3caca5a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "572c3d3778284c0ea3251f5e58b1eac8" is_pre_election: true
I20260812 16:55:50.375125 7287 raft_consensus.cc:2454] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2562853f325488b9035a1f3caca5a53 in term 0.
I20260812 16:55:50.375075 7019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68860c5952844c61a88afbcc1bcabe2c" candidate_uuid: "f2562853f325488b9035a1f3caca5a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" is_pre_election: true
I20260812 16:55:50.375624 7019 raft_consensus.cc:2454] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2562853f325488b9035a1f3caca5a53 in term 0.
I20260812 16:55:50.375883 7419 leader_election.cc:302] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 572c3d3778284c0ea3251f5e58b1eac8, f2562853f325488b9035a1f3caca5a53; no voters:
I20260812 16:55:50.376273 7688 raft_consensus.cc:2781] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:55:50.376430 7688 raft_consensus.cc:482] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:55:50.376565 7688 raft_consensus.cc:3037] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:50.378979 7688 raft_consensus.cc:504] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.379565 7688 leader_election.cc:288] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [CANDIDATE]: Term 1 election: Requested vote from peers 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923), 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517)
I20260812 16:55:50.380033 7287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68860c5952844c61a88afbcc1bcabe2c" candidate_uuid: "f2562853f325488b9035a1f3caca5a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "572c3d3778284c0ea3251f5e58b1eac8"
I20260812 16:55:50.380125 7019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68860c5952844c61a88afbcc1bcabe2c" candidate_uuid: "f2562853f325488b9035a1f3caca5a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
I20260812 16:55:50.380242 7287 raft_consensus.cc:3037] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:50.380384 7019 raft_consensus.cc:3037] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:55:50.382682 7019 raft_consensus.cc:2454] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2562853f325488b9035a1f3caca5a53 in term 1.
I20260812 16:55:50.382743 7287 raft_consensus.cc:2454] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2562853f325488b9035a1f3caca5a53 in term 1.
I20260812 16:55:50.383098 7419 leader_election.cc:302] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe9dc22c1d041ffa92731207ba5ec49, f2562853f325488b9035a1f3caca5a53; no voters:
I20260812 16:55:50.383404 7688 raft_consensus.cc:2781] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:55:50.383880 7688 raft_consensus.cc:686] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 1 LEADER]: Becoming Leader. State: Replica: f2562853f325488b9035a1f3caca5a53, State: Running, Role: LEADER
I20260812 16:55:50.384454 7688 consensus_queue.cc:237] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:55:50.389325 6717 catalog_manager.cc:5495] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2562853f325488b9035a1f3caca5a53 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "f2562853f325488b9035a1f3caca5a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } health_report { overall_health: HEALTHY } } }
I20260812 16:55:50.437732 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.432493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47170 (local address 127.8.195.131:41759)
0812 16:55:50.432677 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.432681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.432699 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:55:50.432790 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.432793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.432871 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.432999 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:55:50.433461 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.434229 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.435712 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.436101 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.436194 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.436683 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:55:50.437222 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:55:50.437334 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.437453 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.437520 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:55:50.446137 7019 raft_consensus.cc:1261] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Refusing update from remote peer f2562853f325488b9035a1f3caca5a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:50.446138 7287 raft_consensus.cc:1261] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Refusing update from remote peer f2562853f325488b9035a1f3caca5a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:55:50.447247 7694 consensus_queue.cc:1035] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:50.447700 7688 consensus_queue.cc:1035] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:55:50.456863 7701 mvcc.cc:205] Tried to move back new op lower bound from 7317724161816563712 to 7317724161579220992. Current Snapshot: MvccSnapshot[applied={T|T < 7317724161816563712}]
I20260812 16:55:50.459126 7709 mvcc.cc:205] Tried to move back new op lower bound from 7317724161816563712 to 7317724161579220992. Current Snapshot: MvccSnapshot[applied={T|T < 7317724161816563712}]
I20260812 16:55:50.473356 7707 mvcc.cc:205] Tried to move back new op lower bound from 7317724161816563712 to 7317724161579220992. Current Snapshot: MvccSnapshot[applied={T|T < 7317724161816563712}]
I20260812 16:55:50.475970 6716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:50.476351 6716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:55:50.480485 6716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Sending DeleteTablet for 3 replicas of tablet 68860c5952844c61a88afbcc1bcabe2c
I20260812 16:55:50.482028 7247 tablet_service.cc:1514] Processing DeleteTablet for tablet 68860c5952844c61a88afbcc1bcabe2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:57068
I20260812 16:55:50.482038 7501 tablet_service.cc:1514] Processing DeleteTablet for tablet 68860c5952844c61a88afbcc1bcabe2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:47154
I20260812 16:55:50.483042 7729 tablet_replica.cc:330] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: stopping tablet replica
I20260812 16:55:50.483080 6976 tablet_service.cc:1514] Processing DeleteTablet for tablet 68860c5952844c61a88afbcc1bcabe2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:55:50 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:51762
I20260812 16:55:50.483240 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 6845
I20260812 16:55:50.483503 7729 raft_consensus.cc:2229] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:50.484066 7729 raft_consensus.cc:2258] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:50.484429 7728 tablet_replica.cc:330] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: stopping tablet replica
I20260812 16:55:50.484946 7728 raft_consensus.cc:2229] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:55:50.485877 7728 raft_consensus.cc:2258] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:50.488787 7729 ts_tablet_manager.cc:1905] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:50.491108 7730 tablet_replica.cc:330] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: stopping tablet replica
I20260812 16:55:50.492713 7728 ts_tablet_manager.cc:1905] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:50.493294 7730 raft_consensus.cc:2229] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:55:50.495648 7730 raft_consensus.cc:2258] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:55:50.497006 7729 ts_tablet_manager.cc:1918] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:50.497258 7729 log.cc:1196] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/68860c5952844c61a88afbcc1bcabe2c
I20260812 16:55:50.497898 7729 ts_tablet_manager.cc:1939] T 68860c5952844c61a88afbcc1bcabe2c P 572c3d3778284c0ea3251f5e58b1eac8: Deleting consensus metadata
I20260812 16:55:50.498929 7730 ts_tablet_manager.cc:1905] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:55:50.500084 6692 catalog_manager.cc:4841] TS 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923): tablet 68860c5952844c61a88afbcc1bcabe2c (table test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]) successfully deleted
I20260812 16:55:50.500816 7728 ts_tablet_manager.cc:1918] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:50.501014 7728 log.cc:1196] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/68860c5952844c61a88afbcc1bcabe2c
I20260812 16:55:50.501607 7728 ts_tablet_manager.cc:1939] T 68860c5952844c61a88afbcc1bcabe2c P f2562853f325488b9035a1f3caca5a53: Deleting consensus metadata
I20260812 16:55:50.503427 6691 catalog_manager.cc:4841] TS f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759): tablet 68860c5952844c61a88afbcc1bcabe2c (table test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]) successfully deleted
I20260812 16:55:50.505842 7730 ts_tablet_manager.cc:1918] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:55:50.506028 7730 log.cc:1196] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/68860c5952844c61a88afbcc1bcabe2c
I20260812 16:55:50.506497 7730 ts_tablet_manager.cc:1939] T 68860c5952844c61a88afbcc1bcabe2c P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting consensus metadata
Aug 12 16:55:50 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:55:50.508402 6692 catalog_manager.cc:4841] TS 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517): tablet 68860c5952844c61a88afbcc1bcabe2c (table test-table [id=ba2c77e9907f43928d41bffa7fd6a17e]) successfully deleted
I20260812 16:55:50.515298 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.484581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.8.195.129:37517)
0812 16:55:50.484785 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.484792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.487070 (+ 2278us) server_negotiation.cc:407] Connection header received
0812 16:55:50.487426 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.487436 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.487544 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.487636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:50.491351 (+ 3715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.492518 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.496107 (+ 3589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.496769 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.496873 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.508879 (+ 12006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:50.508891 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:50.508905 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:50.508940 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:50.512187 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.512981 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.512988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.512991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.513127 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.513628 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.513633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.513635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.513993 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:50.514326 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.514703 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.514937 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:55:50.515951 7085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:50.635268 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 6845
I20260812 16:55:50.669260 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 7109
Aug 12 16:55:50 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:55:50.687448 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.670493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.8.195.130:40923)
0812 16:55:50.670686 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.670692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.670713 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:55:50.670781 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.670787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.670879 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.670994 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:50.672473 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.673626 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.674992 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.675638 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.675777 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.680245 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:50.680268 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:50.680289 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:50.680343 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:50.684259 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.685073 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.685084 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.685088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.685289 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.685671 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.685677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.685681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.686202 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:50.686526 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.686814 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.687062 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:55:50.688184 7363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:50.780655 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 7109
I20260812 16:55:50.808379 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 7385
Aug 12 16:55:50 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:55:50.827139 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:55:50.809308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.8.195.131:41759)
0812 16:55:50.809493 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:55:50.809499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:55:50.809524 (+ 25us) server_negotiation.cc:407] Connection header received
0812 16:55:50.809753 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:55:50.809759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:55:50.809875 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:55:50.810002 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:55:50.811416 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.812239 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.813790 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:55:50.814257 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:55:50.814415 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:55:50.819439 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:55:50.819451 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:55:50.819467 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:55:50.819511 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:55:50.823624 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.824535 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.824546 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.824550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.824784 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:55:50.825243 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:55:50.825249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:55:50.825252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:55:50.825773 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:55:50.826133 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0812 16:55:50.826452 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 16:55:50.826693 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:55:50.828083 7629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:50.955317 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 7385
I20260812 16:55:50.985962 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 6644
I20260812 16:55:50.987226 6786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:55:51.101066 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 6644
I20260812 16:56:11.137281 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--webserver_interface=127.8.195.190
--webserver_port=38391
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:11.501865 7740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:11.502175 7740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:11.502331 7740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:11.509809 7740 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:56:11.509922 7740 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:56:11.510011 7740 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:56:11.510092 7740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:56:11.510190 7740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:11.510309 7740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:11.510385 7740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:56:11.510465 7740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:56:11.520540 7740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:11.521996 7740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:11.523384 7740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:11.550154 7750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:11.550072 7751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:11.550100 7755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:11.551011 7740 server_base.cc:1034] running on GCE node
I20260812 16:56:11.551963 7740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:11.553655 7740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:11.554920 7740 hybrid_clock.cc:648] HybridClock initialized: now 1786553771554861 us; error 47 us; skew 500 ppm
I20260812 16:56:11.555336 7740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:11 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553771, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:11.562980 7740 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:56:11.565019 7740 webserver.cc:460] Webserver started at http://127.8.195.190:38391/ using document root <none> and password file <none>
I20260812 16:56:11.566099 7740 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:11.566277 7740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:11.572942 7740 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 16:56:11.576150 7768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:11.577661 7740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:56:11.577875 7740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "6b1fe827500045ada311fd9731923601"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "9ecb4a9ed998a3e1015b2958745869a5"
server_key_iv: "5b21d0839e2fbd26fd14159cbc32fd4c"
server_key_version: "encryptionkey@0"
I20260812 16:56:11.578999 7740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:11.599323 7740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:11.604326 7740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:11.604684 7740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:11.625731 7740 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37423
I20260812 16:56:11.625765 7864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37423 every 8 connection(s)
I20260812 16:56:11.627509 7740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:56:11.631990 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 7740
I20260812 16:56:11.632885 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:37517
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=43181
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:56:11.635156 7865 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:56:11.638741 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap starting.
I20260812 16:56:11.664932 7865 log.cc:824] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:11.682700 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:56:11.683462 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap complete.
I20260812 16:56:11.697738 7865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:11.699014 7865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1fe827500045ada311fd9731923601, State: Initialized, Role: FOLLOWER
I20260812 16:56:11.699836 7865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:11.700172 7865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:56:11.700317 7865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:56:11.700528 7865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 1 FOLLOWER]: Advancing to term 2
I20260812 16:56:11.707027 7865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:11.707821 7865 leader_election.cc:302] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b1fe827500045ada311fd9731923601; no voters:
I20260812 16:56:11.708561 7865 leader_election.cc:288] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 16:56:11.708741 7880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 16:56:11.710219 7880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 LEADER]: Becoming Leader. State: Replica: 6b1fe827500045ada311fd9731923601, State: Running, Role: LEADER
I20260812 16:56:11.710965 7865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:56:11.710939 7880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:11.713692 7883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b1fe827500045ada311fd9731923601. Latest consensus state: current_term: 2 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:56:11.714133 7883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:11.713953 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:56:11.714259 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:11.726636 7890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:56:11.735508 7890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:56:11.737114 7890 catalog_manager.cc:1259] Loaded cluster ID: dcb0331b8a574e959aa7f53a7b77323d
I20260812 16:56:11.737356 7890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:56:11.741720 7890 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:56:11.743566 7890 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Loaded TSK: 0
I20260812 16:56:11.768059 7917 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:56:11.779593 7890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Generated new TSK 1
I20260812 16:56:11.780040 7890 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:56:12.036324 7871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:12.036664 7871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:12.036818 7871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:12.044241 7871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:12.044353 7871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:12.044523 7871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:56:12.054576 7871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:37517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=43181
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:12.055888 7871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:12.057291 7871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:12.072405 7933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:12.072393 7934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:12.072531 7936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:12.073702 7871 server_base.cc:1034] running on GCE node
I20260812 16:56:12.074430 7871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:12.075578 7871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:12.076843 7871 hybrid_clock.cc:648] HybridClock initialized: now 1786553772076782 us; error 45 us; skew 500 ppm
I20260812 16:56:12.077245 7871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:12 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:12.081318 7871 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:56:12.082995 7871 webserver.cc:460] Webserver started at http://127.8.195.129:43181/ using document root <none> and password file <none>
I20260812 16:56:12.083741 7871 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:12.083931 7871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:12.089926 7871 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 16:56:12.092973 7962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.094482 7871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:56:12.094663 7871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "293b377314ea40293a09d71f7560e197"
server_key_iv: "dce4bf05adede73239fd949b8bac069c"
server_key_version: "encryptionkey@0"
I20260812 16:56:12.095672 7871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:12.109364 7871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:12.114312 7871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:12.114622 7871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:12.115688 7871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:12.117077 7871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:12.117230 7871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.117368 7871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:12.117457 7871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.152230 7871 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:37517
I20260812 16:56:12.152328 8215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:37517 every 8 connection(s)
I20260812 16:56:12.153857 7871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:56:12.162309 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 7871
I20260812 16:56:12.163259 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:40923
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=33753
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:56:12 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:12.181344 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:12.156826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34485 (local address 127.8.195.190:37423)
0812 16:56:12.157504 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:12.157515 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:12.158261 (+ 746us) server_negotiation.cc:407] Connection header received
0812 16:56:12.160460 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:12.160474 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:12.160821 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:12.161136 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:12.162764 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.164956 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.166692 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.167830 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.168046 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:12.171940 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:12.171953 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:12.171976 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:12.172039 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:12.176341 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:12.177489 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:12.177498 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:12.177503 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:12.177679 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:12.178244 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:12.178247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:12.178249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:12.178762 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:12.178922 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:12.179932 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0812 16:56:12.180282 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":331,"threads_started":1}
I20260812 16:56:12.181761 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:12.157035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:34485)
0812 16:56:12.158023 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:12.158079 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:12.159863 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:12.161332 (+ 1469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:12.161340 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:12.161887 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:12.162491 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:12.162511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.165234 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.165239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:12.166493 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:12.166506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.168070 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.168076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:12.168270 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:12.169012 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:12.169035 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:12.171606 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:12.176517 (+ 4911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:12.176530 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:12.176551 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:12.177320 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:12.177808 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:12.177824 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:12.177831 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:12.178098 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:12.178916 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:12.178928 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:12.179378 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 16:56:12.179710 (+ 332us) client_negotiation.cc:240] Negotiation successful
0812 16:56:12.180357 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":394,"threads_started":1}
I20260812 16:56:12.184880 8216 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:56:12.185266 8216 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:12.186038 8216 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:56:12.188349 7796 ts_manager.cc:194] Registered new tserver with Master: 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517)
I20260812 16:56:12.190459 7796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:34485
W20260812 16:56:12.516644 8222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:12.516999 8222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:12.517349 8222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:12.524739 8222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:12.524904 8222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:12.525096 8222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:56:12.535272 8222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:40923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=33753
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:12.536702 8222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:12.538235 8222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:12.552433 8234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:12.552464 8233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:12.552654 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:12.553282 8222 server_base.cc:1034] running on GCE node
I20260812 16:56:12.553996 8222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:12.555080 8222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:12.556324 8222 hybrid_clock.cc:648] HybridClock initialized: now 1786553772556269 us; error 50 us; skew 500 ppm
I20260812 16:56:12.556663 8222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:12 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:12.559973 8222 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:56:12.561645 8222 webserver.cc:460] Webserver started at http://127.8.195.130:33753/ using document root <none> and password file <none>
I20260812 16:56:12.562366 8222 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:12.562536 8222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:12.568039 8222 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 16:56:12.570765 8254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.572098 8222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:56:12.572291 8222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "572c3d3778284c0ea3251f5e58b1eac8"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "d9274b48d4b4aa2ed66ac35679d77bea"
server_key_iv: "7a5eaa363b4100090a9e39a757787f19"
server_key_version: "encryptionkey@0"
I20260812 16:56:12.573412 8222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:12.598285 8222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:12.602615 8222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:12.602916 8222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:12.604090 8222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:12.605628 8222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:12.605784 8222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.605919 8222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:12.606014 8222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:12.642612 8222 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40923
I20260812 16:56:12.642678 8489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40923 every 8 connection(s)
I20260812 16:56:12.644066 8222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:56:12.650071 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 8222
I20260812 16:56:12.650902 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:41759
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=45865
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 16:56:12 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553772, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:12.668721 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:12.646678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:38801 (local address 127.8.195.190:37423)
0812 16:56:12.646872 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:12.646876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:12.647871 (+ 995us) server_negotiation.cc:407] Connection header received
0812 16:56:12.649841 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:12.649846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:12.649945 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:12.650034 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:12.652114 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.652801 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.655306 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.655823 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.655937 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:12.660725 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:12.660734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:12.660742 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:12.660793 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:12.664667 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:12.665744 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:12.665752 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:12.665757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:12.665928 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:12.666516 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:12.666523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:12.666527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:12.666878 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:12.667033 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:12.668026 (+ 993us) server_negotiation.cc:299] Negotiation successful
0812 16:56:12.668327 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:56:12.670280 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:12.647016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:38801)
0812 16:56:12.647695 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:12.647732 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:12.649494 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:12.650319 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:12.650331 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:12.651110 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:12.651916 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:12.651945 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.652970 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.652978 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:12.655148 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:12.655166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:12.656076 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:12.656087 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:12.656431 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:12.657333 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:12.657373 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:12.660399 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:12.664846 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:12.664856 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:12.664872 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:12.665548 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:12.666066 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:12.666071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:12.666074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:12.666365 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:12.667018 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:12.667026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:12.667517 (+ 491us) client_negotiation.cc:769] Sending connection context
0812 16:56:12.667873 (+ 356us) client_negotiation.cc:240] Negotiation successful
0812 16:56:12.668421 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":269,"threads_started":1}
I20260812 16:56:12.671887 8492 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:56:12.672254 8492 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:12.673300 8492 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:56:12.675096 7796 ts_manager.cc:194] Registered new tserver with Master: 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923)
I20260812 16:56:12.675947 7796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:38801
W20260812 16:56:13.004200 8496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:13.004606 8496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:13.004746 8496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:13.012243 8496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:13.012393 8496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:13.012591 8496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:56:13.022686 8496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=45865
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:13.024053 8496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:13.025588 8496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:13.040091 8504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:13.040226 8501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:13.040587 8507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:13.042178 8496 server_base.cc:1034] running on GCE node
I20260812 16:56:13.042843 8496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:13.043828 8496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:13.045063 8496 hybrid_clock.cc:648] HybridClock initialized: now 1786553773045020 us; error 55 us; skew 500 ppm
I20260812 16:56:13.045418 8496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:13 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:13.048846 8496 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:56:13.050400 8496 webserver.cc:460] Webserver started at http://127.8.195.131:45865/ using document root <none> and password file <none>
I20260812 16:56:13.051138 8496 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:13.051308 8496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:13.056816 8496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:56:13.059654 8530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:13.060974 8496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 16:56:13.061272 8496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f2562853f325488b9035a1f3caca5a53"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "fe6786ac79dd9eef8804cb1187e2199a"
server_key_iv: "967bdfcd413d56563f9a3fe89b7ba53d"
server_key_version: "encryptionkey@0"
I20260812 16:56:13.062337 8496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:13.077818 8496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:13.082757 8496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:13.083055 8496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:13.084224 8496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:13.085700 8496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:13.085840 8496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:13.085971 8496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:13.086072 8496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:13.122938 8496 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:41759
I20260812 16:56:13.123034 8752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:41759 every 8 connection(s)
I20260812 16:56:13.124507 8496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:56:13.129716 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 8496
Aug 12 16:56:13 dist-test-slave-z8x9 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553773, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:13.147910 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.127449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:57315 (local address 127.8.195.190:37423)
0812 16:56:13.127644 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.127649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.129006 (+ 1357us) server_negotiation.cc:407] Connection header received
0812 16:56:13.131179 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.131189 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.131299 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.131395 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:13.133232 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.134034 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.135677 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.136104 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.136239 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.140533 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:13.140538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:13.140541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:13.140587 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:13.143700 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.144740 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.144748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.144757 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.144904 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.145507 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.145511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.145513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.145942 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:13.146050 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.147173 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.147475 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":70}
I20260812 16:56:13.148773 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.130869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.8.195.190:37423)
0812 16:56:13.131746 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.131755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.131776 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:56:13.131848 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.131851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.131903 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.132026 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:13.133930 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.134892 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.136402 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.136960 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.137113 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.140465 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:13.140472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:13.140478 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:13.140529 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:13.144465 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.145308 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.145314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.145318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.145476 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.145947 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.145951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.145953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.146455 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:13.146764 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.147170 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.148308 (+ 1138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":754,"thread_start_us":604,"threads_started":1}
I20260812 16:56:13.149528 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.127880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:57315)
0812 16:56:13.128785 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.128832 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.130809 (+ 1977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.131929 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.131937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.132457 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:13.133018 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.133035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.134216 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.134221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.135524 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.135536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.136348 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.136355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.136592 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.137450 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:13.137481 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:13.140263 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:13.143867 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:13.143879 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:13.143903 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:13.144615 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:13.145032 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:13.145038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:13.145042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:13.145366 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:13.146184 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:13.146193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:13.146700 (+ 507us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.147305 (+ 605us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.147945 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":393,"threads_started":1}
I20260812 16:56:13.151222 8753 heartbeater.cc:347] Connected to a master server at 127.8.195.190:37423
I20260812 16:56:13.151593 8753 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:13.152398 8753 heartbeater.cc:511] Master 127.8.195.190:37423 requested a full tablet report, sending...
I20260812 16:56:13.153997 7793 ts_manager.cc:194] Registered new tserver with Master: f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759)
I20260812 16:56:13.154983 7793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:57315
I20260812 16:56:13.166839 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:56:13.174629 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.167994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.8.195.190:37423)
0812 16:56:13.168237 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.168244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.168892 (+ 648us) server_negotiation.cc:407] Connection header received
0812 16:56:13.169106 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.169110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.169363 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.169510 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:56:13.169981 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.170740 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.172400 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.172887 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.172993 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.173741 (+ 748us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:56:13.174041 (+ 300us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 16:56:13.174408 (+ 367us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35214: token expired
Metrics: {"server-negotiator.queue_time_us":95}
W20260812 16:56:13.174763 8764 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35214: token expired
W20260812 16:56:13.175521 7645 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.195.190:37423)
I20260812 16:56:13.190874 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.176196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.8.195.190:37423)
0812 16:56:13.176407 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.176412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.176492 (+ 80us) server_negotiation.cc:407] Connection header received
0812 16:56:13.176721 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.176726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.176795 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.176917 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:13.178601 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.179388 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.180843 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.181359 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.181506 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.184720 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:13.184726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:13.184731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:13.184772 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:13.187723 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.188549 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.188555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.188557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.188717 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.189233 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.189238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.189240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.189651 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:13.189879 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.190255 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.190485 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 16:56:13.193660 8216 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:56:13.195217 7793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:56:13.203997 7793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:56:13.231947 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.217925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54224)
0812 16:56:13.218890 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.218945 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.219127 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.220643 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.220655 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.220702 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:13.221518 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.221542 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.224356 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.224366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.227541 (+ 3175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.227565 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.230517 (+ 2952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.230533 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.230905 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.230990 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.231137 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.231414 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":502,"threads_started":1}
I20260812 16:56:13.235605 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.218133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.8.195.129:37517)
0812 16:56:13.219847 (+ 1714us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.219856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.219889 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:56:13.219996 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.220005 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.220337 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.221846 (+ 1509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:13.222158 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.224144 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.227849 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.230065 (+ 2216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.232707 (+ 2642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.233084 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.234974 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.235221 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1502,"thread_start_us":1303,"threads_started":1}
I20260812 16:56:13.240674 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.219343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40923 (local address 127.0.0.1:53288)
0812 16:56:13.232307 (+ 12964us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.232334 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.232475 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.234457 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.234466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.234509 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:13.234729 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.234742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.236878 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.236884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.238431 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.238445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.239804 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.239811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.239996 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.240038 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.240164 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.240334 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12819,"thread_start_us":15139,"threads_started":1}
I20260812 16:56:13.240696 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.219919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.8.195.130:40923)
0812 16:56:13.233341 (+ 13422us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.233350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.233386 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:56:13.233498 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.233507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.233829 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.234072 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:13.234932 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.236665 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.238613 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.239603 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.239779 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.240006 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.240210 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.240378 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13215,"thread_start_us":13078,"threads_started":1}
I20260812 16:56:13.240320 8054 tablet_service.cc:1467] Processing CreateTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad (DEFAULT_TABLE table=test-table [id=fc9b2eb199a34fa6b8a09ba57266e497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:13.242244 8054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3bb2dd0b0c54c32b96e19f5bd206aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:13.243090 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.219770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:41759 (local address 127.0.0.1:54302)
0812 16:56:13.233723 (+ 13953us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.233744 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.234030 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.234772 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.234778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.234808 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:13.235144 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.235152 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.237449 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.237455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.239590 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.239606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.242201 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.242208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.242422 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.242457 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.242554 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.242705 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13831,"thread_start_us":12811,"threads_started":1}
I20260812 16:56:13.244069 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.220149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.8.195.131:41759)
0812 16:56:13.233341 (+ 13192us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.233350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.233908 (+ 558us) server_negotiation.cc:407] Connection header received
0812 16:56:13.234218 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.234229 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.234482 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.234690 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:13.235330 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.237280 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.239879 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.241626 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.241778 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.242019 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.243522 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.243714 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12945,"thread_start_us":13200,"threads_started":1}
I20260812 16:56:13.244783 8351 tablet_service.cc:1467] Processing CreateTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad (DEFAULT_TABLE table=test-table [id=fc9b2eb199a34fa6b8a09ba57266e497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:13.246301 8351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3bb2dd0b0c54c32b96e19f5bd206aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:13.248075 8618 tablet_service.cc:1467] Processing CreateTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad (DEFAULT_TABLE table=test-table [id=fc9b2eb199a34fa6b8a09ba57266e497]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:13.250001 8618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3bb2dd0b0c54c32b96e19f5bd206aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:13.255484 8784 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Bootstrap starting.
I20260812 16:56:13.259395 8784 tablet_bootstrap.cc:654] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:13.260651 8784 log.cc:824] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:13.263161 8786 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Bootstrap starting.
I20260812 16:56:13.263854 8784 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: No bootstrap required, opened a new log
I20260812 16:56:13.264156 8784 ts_tablet_manager.cc:1397] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 16:56:13.266052 8787 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Bootstrap starting.
I20260812 16:56:13.267153 8786 tablet_bootstrap.cc:654] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:13.268375 8786 log.cc:824] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:13.269905 8787 tablet_bootstrap.cc:654] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:13.271164 8787 log.cc:824] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:13.272099 8786 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: No bootstrap required, opened a new log
I20260812 16:56:13.272428 8786 ts_tablet_manager.cc:1397] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 16:56:13.274731 8787 tablet_bootstrap.cc:492] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: No bootstrap required, opened a new log
I20260812 16:56:13.275029 8787 ts_tablet_manager.cc:1397] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260812 16:56:13.275456 8784 raft_consensus.cc:348] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.275959 8784 raft_consensus.cc:374] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:13.276073 8784 raft_consensus.cc:729] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe9dc22c1d041ffa92731207ba5ec49, State: Initialized, Role: FOLLOWER
I20260812 16:56:13.276787 8784 consensus_queue.cc:260] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.278826 8784 ts_tablet_manager.cc:1428] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260812 16:56:13.281534 8786 raft_consensus.cc:348] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.282016 8786 raft_consensus.cc:374] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:13.282135 8786 raft_consensus.cc:729] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572c3d3778284c0ea3251f5e58b1eac8, State: Initialized, Role: FOLLOWER
I20260812 16:56:13.283216 8786 consensus_queue.cc:260] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.284418 8492 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:56:13.286585 8786 ts_tablet_manager.cc:1428] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260812 16:56:13.286782 8787 raft_consensus.cc:348] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.287254 8787 raft_consensus.cc:374] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:13.287374 8787 raft_consensus.cc:729] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2562853f325488b9035a1f3caca5a53, State: Initialized, Role: FOLLOWER
I20260812 16:56:13.288033 8787 consensus_queue.cc:260] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.289418 8753 heartbeater.cc:503] Master 127.8.195.190:37423 was elected leader, sending a full tablet report...
I20260812 16:56:13.289877 8787 ts_tablet_manager.cc:1428] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260812 16:56:13.358496 8793 raft_consensus.cc:482] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:56:13.358960 8793 raft_consensus.cc:504] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.362079 8793 leader_election.cc:288] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517), f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759)
W20260812 16:56:13.376674 8756 tablet.cc:2369] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:13.380045 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.363092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:38737 (local address 127.8.195.129:37517)
0812 16:56:13.363500 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.363508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.364436 (+ 928us) server_negotiation.cc:407] Connection header received
0812 16:56:13.364731 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.364738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.364889 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.365059 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:13.366828 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.368824 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.374807 (+ 5983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.377314 (+ 2507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.377676 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.377871 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.378837 (+ 966us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.379290 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 16:56:13.380245 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.362918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:33665 (local address 127.8.195.131:41759)
0812 16:56:13.363385 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.363408 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.364148 (+ 740us) server_negotiation.cc:407] Connection header received
0812 16:56:13.364557 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.364569 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.364740 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.364954 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:13.366577 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.368804 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.374790 (+ 5986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.377565 (+ 2775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.377880 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.378231 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.379215 (+ 984us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.379651 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 16:56:13.380529 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.362576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:41759 (local address 127.8.195.130:33665)
0812 16:56:13.363923 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.363977 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.364254 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.365014 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.365035 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.365090 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:13.366283 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.366327 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.369098 (+ 2771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.369113 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.374276 (+ 5163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.374324 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.378278 (+ 3954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.378299 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.378838 (+ 539us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.378914 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.379109 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.379501 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":399,"thread_start_us":836,"threads_started":1}
I20260812 16:56:13.380839 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.362633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.8.195.130:38737)
0812 16:56:13.364283 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.364310 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:13.364505 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:13.365089 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:13.365104 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:13.365135 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:13.366421 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.366438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.369096 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.369113 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.374314 (+ 5201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:13.374326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.377741 (+ 3415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.377750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:13.378257 (+ 507us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.378509 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 16:56:13.378796 (+ 287us) client_negotiation.cc:240] Negotiation successful
0812 16:56:13.379091 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"mutex_wait_us":420,"thread_start_us":1161,"threads_started":1}
I20260812 16:56:13.382551 8657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3bb2dd0b0c54c32b96e19f5bd206aad" candidate_uuid: "572c3d3778284c0ea3251f5e58b1eac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2562853f325488b9035a1f3caca5a53" is_pre_election: true
I20260812 16:56:13.383133 8115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3bb2dd0b0c54c32b96e19f5bd206aad" candidate_uuid: "572c3d3778284c0ea3251f5e58b1eac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" is_pre_election: true
I20260812 16:56:13.383424 8657 raft_consensus.cc:2454] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572c3d3778284c0ea3251f5e58b1eac8 in term 0.
I20260812 16:56:13.384080 8115 raft_consensus.cc:2454] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 572c3d3778284c0ea3251f5e58b1eac8 in term 0.
I20260812 16:56:13.384569 8255 leader_election.cc:302] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 572c3d3778284c0ea3251f5e58b1eac8, f2562853f325488b9035a1f3caca5a53; no voters:
I20260812 16:56:13.385411 8793 raft_consensus.cc:2781] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:56:13.385653 8793 raft_consensus.cc:482] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:56:13.385867 8793 raft_consensus.cc:3037] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:13.390151 8793 raft_consensus.cc:504] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
I20260812 16:56:13.391335 8793 leader_election.cc:288] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [CANDIDATE]: Term 1 election: Requested vote from peers 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517), f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759)
I20260812 16:56:13.392185 8115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3bb2dd0b0c54c32b96e19f5bd206aad" candidate_uuid: "572c3d3778284c0ea3251f5e58b1eac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
I20260812 16:56:13.392207 8657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3bb2dd0b0c54c32b96e19f5bd206aad" candidate_uuid: "572c3d3778284c0ea3251f5e58b1eac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2562853f325488b9035a1f3caca5a53"
I20260812 16:56:13.392601 8115 raft_consensus.cc:3037] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:13.392617 8657 raft_consensus.cc:3037] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 0 FOLLOWER]: Advancing to term 1
W20260812 16:56:13.396416 8494 tablet.cc:2369] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:13.396924 8657 raft_consensus.cc:2454] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572c3d3778284c0ea3251f5e58b1eac8 in term 1.
I20260812 16:56:13.397390 8115 raft_consensus.cc:2454] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572c3d3778284c0ea3251f5e58b1eac8 in term 1.
I20260812 16:56:13.397747 8255 leader_election.cc:302] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 572c3d3778284c0ea3251f5e58b1eac8, f2562853f325488b9035a1f3caca5a53; no voters:
I20260812 16:56:13.398468 8793 raft_consensus.cc:2781] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:56:13.399783 8793 raft_consensus.cc:686] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 LEADER]: Becoming Leader. State: Replica: 572c3d3778284c0ea3251f5e58b1eac8, State: Running, Role: LEADER
I20260812 16:56:13.400640 8793 consensus_queue.cc:237] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } }
W20260812 16:56:13.406795 8217 tablet.cc:2369] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:13.410297 7793 catalog_manager.cc:5495] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130). New cstate: current_term: 1 leader_uuid: "572c3d3778284c0ea3251f5e58b1eac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "572c3d3778284c0ea3251f5e58b1eac8" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 } health_report { overall_health: UNKNOWN } } }
I20260812 16:56:13.442140 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.431403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.8.195.130:40923)
0812 16:56:13.431835 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.431853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.431910 (+ 57us) server_negotiation.cc:407] Connection header received
0812 16:56:13.432398 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.432402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.432568 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.432829 (+ 261us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:56:13.433602 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.435081 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.437750 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.438462 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.438722 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.439805 (+ 1083us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:56:13.440687 (+ 882us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:56:13.441048 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.441279 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.441542 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260812 16:56:13.456777 8115 raft_consensus.cc:1261] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Refusing update from remote peer 572c3d3778284c0ea3251f5e58b1eac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:13.456777 8657 raft_consensus.cc:1261] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Refusing update from remote peer 572c3d3778284c0ea3251f5e58b1eac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:13.458633 8793 consensus_queue.cc:1035] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2562853f325488b9035a1f3caca5a53" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:56:13.459182 8800 consensus_queue.cc:1035] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe9dc22c1d041ffa92731207ba5ec49" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 37517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:56:13.483057 8804 mvcc.cc:205] Tried to move back new op lower bound from 7317724256061734912 to 7317724255857020928. Current Snapshot: MvccSnapshot[applied={T|T < 7317724256061734912}]
I20260812 16:56:13.515166 7793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:13.515602 7793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:13.521826 7793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Sending DeleteTablet for 3 replicas of tablet d3bb2dd0b0c54c32b96e19f5bd206aad
I20260812 16:56:13.523931 8054 tablet_service.cc:1514] Processing DeleteTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:13 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:54224
I20260812 16:56:13.524077 8618 tablet_service.cc:1514] Processing DeleteTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:13 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:54302
I20260812 16:56:13.524097 8351 tablet_service.cc:1514] Processing DeleteTablet for tablet d3bb2dd0b0c54c32b96e19f5bd206aad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:13 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:53288
I20260812 16:56:13.527473 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 7871
I20260812 16:56:13.535447 8833 tablet_replica.cc:330] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: stopping tablet replica
I20260812 16:56:13.536546 8833 raft_consensus.cc:2229] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:56:13.538077 8833 raft_consensus.cc:2258] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:13.540298 8830 tablet_replica.cc:330] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: stopping tablet replica
I20260812 16:56:13.540539 8832 tablet_replica.cc:330] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: stopping tablet replica
I20260812 16:56:13.541224 8832 raft_consensus.cc:2229] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:13.541136 8830 raft_consensus.cc:2229] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:13.542119 8830 raft_consensus.cc:2258] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:13.542107 8832 raft_consensus.cc:2258] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:13.546442 8832 ts_tablet_manager.cc:1905] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:13.546720 8830 ts_tablet_manager.cc:1905] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:13.550513 8833 ts_tablet_manager.cc:1905] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:13.559873 8832 ts_tablet_manager.cc:1918] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:13.560240 8832 log.cc:1196] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/d3bb2dd0b0c54c32b96e19f5bd206aad
I20260812 16:56:13.561316 8832 ts_tablet_manager.cc:1939] T d3bb2dd0b0c54c32b96e19f5bd206aad P f2562853f325488b9035a1f3caca5a53: Deleting consensus metadata
I20260812 16:56:13.564705 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.529910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.8.195.129:37517)
0812 16:56:13.530265 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.530272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.530315 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:56:13.530424 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.530430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.530619 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.530848 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:13.535166 (+ 4318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.537036 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.541030 (+ 3994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.542504 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.543262 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.549786 (+ 6524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:13.549821 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:13.549863 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:13.549969 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:13.557284 (+ 7315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.559001 (+ 1717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.559024 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.559028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.559363 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.560259 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.560276 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.560279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.561356 (+ 1077us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:13.562029 (+ 673us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.563392 (+ 1363us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.563878 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260812 16:56:13.565480 7770 catalog_manager.cc:4841] TS f2562853f325488b9035a1f3caca5a53 (127.8.195.131:41759): tablet d3bb2dd0b0c54c32b96e19f5bd206aad (table test-table [id=fc9b2eb199a34fa6b8a09ba57266e497]) successfully deleted
I20260812 16:56:13.566262 8210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:13.571561 8833 ts_tablet_manager.cc:1918] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:13.572073 8833 log.cc:1196] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/d3bb2dd0b0c54c32b96e19f5bd206aad
I20260812 16:56:13.573652 8833 ts_tablet_manager.cc:1939] T d3bb2dd0b0c54c32b96e19f5bd206aad P 572c3d3778284c0ea3251f5e58b1eac8: Deleting consensus metadata
I20260812 16:56:13.578398 7771 catalog_manager.cc:4841] TS 572c3d3778284c0ea3251f5e58b1eac8 (127.8.195.130:40923): tablet d3bb2dd0b0c54c32b96e19f5bd206aad (table test-table [id=fc9b2eb199a34fa6b8a09ba57266e497]) successfully deleted
I20260812 16:56:13.747862 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 7871
W20260812 16:56:13.803812 7771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:37517 (error 108)
I20260812 16:56:13.804244 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 8222
I20260812 16:56:13.808730 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.806886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54240)
0812 16:56:13.808309 (+ 1423us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.808482 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":1035,"threads_started":1}
W20260812 16:56:13.809290 7771 catalog_manager.cc:4584] TS 8fe9dc22c1d041ffa92731207ba5ec49 (127.8.195.129:37517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3bb2dd0b0c54c32b96e19f5bd206aad: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
I20260812 16:56:13.838244 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.806427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.8.195.130:40923)
0812 16:56:13.806758 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:13.806768 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:13.806801 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:56:13.806928 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:13.806935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:13.807095 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:13.807325 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:13.810944 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.812375 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.814608 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:13.815321 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:13.815575 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:13.826663 (+ 11088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:13.826712 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:13.826738 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:13.826816 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:13.832586 (+ 5770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.834161 (+ 1575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.834173 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.834175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.834461 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:13.835216 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:13.835241 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:13.835244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:13.835806 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:13.836509 (+ 703us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:13.837080 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 16:56:13.837568 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 16:56:13.839293 8480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:13.864174 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.863031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54252)
0812 16:56:13.863500 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.863780 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260812 16:56:13.866703 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.865827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54258)
0812 16:56:13.866097 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.866324 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 16:56:13.928529 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.927601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54262)
0812 16:56:13.927962 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.928175 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 16:56:13.930533 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:13.929718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54264)
0812 16:56:13.929993 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:13.930210 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 16:56:14.006274 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 8222
I20260812 16:56:14.019268 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:14.018478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54280)
0812 16:56:14.018832 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:14.018993 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 16:56:14.021492 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:14.020651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54286)
0812 16:56:14.020993 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:14.021123 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260812 16:56:14.051831 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 8496
I20260812 16:56:14.076644 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:14.053058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54306 (local address 127.8.195.131:41759)
0812 16:56:14.054015 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:14.054029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:14.054046 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:56:14.054111 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:14.054123 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:14.054220 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:14.054441 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:14.056811 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:14.058091 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:14.060370 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:14.061099 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:14.061423 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:14.066172 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:14.066222 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:14.066248 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:14.066427 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:14.071683 (+ 5256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:14.072900 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:14.072916 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:14.072919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:14.073267 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:14.074026 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:14.074030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:14.074032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:14.074683 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:14.075038 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:14.075627 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 16:56:14.075956 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":493,"threads_started":1}
I20260812 16:56:14.077911 8744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:14.195087 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:14.194282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54298)
0812 16:56:14.194655 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:14.194839 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260812 16:56:14.196884 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:14.196292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:37517 (local address 127.0.0.1:54314)
0812 16:56:14.196555 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:14.196661 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 16:56:14.211198 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 8496
I20260812 16:56:14.255925 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 7740
I20260812 16:56:14.258025 7854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:14.382071 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 7740
I20260812 16:56:34.418509 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--webserver_interface=127.8.195.190
--webserver_port=38391
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:37423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:34.785734 8851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:34.786059 8851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:34.786254 8851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:34.793700 8851 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 16:56:34.793816 8851 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 16:56:34.793856 8851 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 16:56:34.793969 8851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:56:34.794047 8851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:34.794162 8851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:34.794231 8851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:56:34.794342 8851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:56:34.804495 8851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:37423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:37423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:34.805920 8851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:34.807322 8851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:34.819975 8858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:34.820032 8861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:34.820008 8859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:34.821149 8851 server_base.cc:1034] running on GCE node
I20260812 16:56:34.822099 8851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:34.823660 8851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:34.824884 8851 hybrid_clock.cc:648] HybridClock initialized: now 1786553794824831 us; error 34 us; skew 500 ppm
I20260812 16:56:34.825286 8851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:34 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553794, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:34.832854 8851 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:56:34.834544 8851 webserver.cc:460] Webserver started at http://127.8.195.190:38391/ using document root <none> and password file <none>
I20260812 16:56:34.835273 8851 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:34.835404 8851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:34.841022 8851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:56:34.843969 8875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:34.845330 8851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:56:34.845517 8851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "6b1fe827500045ada311fd9731923601"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "9ecb4a9ed998a3e1015b2958745869a5"
server_key_iv: "5b21d0839e2fbd26fd14159cbc32fd4c"
server_key_version: "encryptionkey@0"
I20260812 16:56:34.846510 8851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:34.872857 8851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:34.877868 8851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:34.878190 8851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:34.898089 8851 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:37423
I20260812 16:56:34.898128 8988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:37423 every 8 connection(s)
I20260812 16:56:34.899663 8851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:56:34.905563 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 8851
I20260812 16:56:34.906474 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:37517
--local_ip_for_outbound_sockets=127.8.195.129
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=43181
--webserver_interface=127.8.195.129
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:56:34.906726 8989 sys_catalog.cc:263] Verifying existing consensus state
I20260812 16:56:34.910017 8989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap starting.
I20260812 16:56:34.935247 8989 log.cc:824] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:34.958344 8989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 16:56:34.959079 8989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Bootstrap complete.
I20260812 16:56:34.972864 8989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:34.974313 8989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1fe827500045ada311fd9731923601, State: Initialized, Role: FOLLOWER
I20260812 16:56:34.975107 8989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:34.975421 8989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:56:34.975564 8989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:56:34.975761 8989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 2 FOLLOWER]: Advancing to term 3
I20260812 16:56:34.981961 8989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:34.982578 8989 leader_election.cc:302] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b1fe827500045ada311fd9731923601; no voters:
I20260812 16:56:34.983287 8989 leader_election.cc:288] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 16:56:34.983453 8993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 16:56:34.984685 8993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [term 3 LEADER]: Becoming Leader. State: Replica: 6b1fe827500045ada311fd9731923601, State: Running, Role: LEADER
I20260812 16:56:34.985338 8993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } }
I20260812 16:56:34.985814 8989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:56:34.988144 8996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:56:34.988144 8997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b1fe827500045ada311fd9731923601. Latest consensus state: current_term: 3 leader_uuid: "6b1fe827500045ada311fd9731923601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1fe827500045ada311fd9731923601" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 37423 } } }
I20260812 16:56:34.988590 8997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:34.988592 8996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601 [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:34.999161 9001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:56:35.008214 9001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:56:35.009859 9001 catalog_manager.cc:1259] Loaded cluster ID: dcb0331b8a574e959aa7f53a7b77323d
I20260812 16:56:35.010026 9001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:56:35.014294 9001 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:56:35.015995 9001 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Loaded TSK: 0
I20260812 16:56:35.016237 9001 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b1fe827500045ada311fd9731923601: Loaded TSK: 1
F20260812 16:56:35.017505 9001 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc262abb9ad google::LogMessage::Fail() at ??:0
@ 0x7fc262abe88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc262abb4de google::LogMessage::Flush() at ??:0
@ 0x7fc262abca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc26681f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc26680e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc2784ee1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc2784ed783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc2785792a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fc2785790c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc2785c35e9 std::function<>::operator()() at ??:0
@ 0x7fc2784f1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc2784efc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc278577610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc278577439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc265707345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc26570a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc26570a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc2656e5eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc2689bf6db start_thread at ??:0
@ 0x7fc26195561f clone at ??:0
W20260812 16:56:35.349712 8991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:35.350108 8991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:35.350310 8991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:35.357769 8991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:35.357877 8991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:35.358036 8991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:56:35.367966 8991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:37517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=43181
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:35.369369 8991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:35.370688 8991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:35.384559 9081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:35.384590 9080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:35.384517 9086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:35.385128 8991 server_base.cc:1034] running on GCE node
I20260812 16:56:35.386008 8991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:35.387041 8991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:35.388306 8991 hybrid_clock.cc:648] HybridClock initialized: now 1786553795388246 us; error 79 us; skew 500 ppm
I20260812 16:56:35.388652 8991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:35 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:35.391996 8991 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:56:35.393476 8991 webserver.cc:460] Webserver started at http://127.8.195.129:43181/ using document root <none> and password file <none>
I20260812 16:56:35.394150 8991 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:35.394289 8991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:35.399691 8991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260812 16:56:35.402762 9095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:35.404062 8991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:56:35.404240 8991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "8fe9dc22c1d041ffa92731207ba5ec49"
format_stamp: "Formatted at 2026-08-12 16:55:48 on dist-test-slave-z8x9"
server_key: "293b377314ea40293a09d71f7560e197"
server_key_iv: "dce4bf05adede73239fd949b8bac069c"
server_key_version: "encryptionkey@0"
I20260812 16:56:35.405274 8991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:35.440857 8991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:35.445238 8991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:35.445538 8991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:35.446621 8991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:35.448560 9108 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 16:56:35.454622 9108 ts_tablet_manager.cc:1852] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 16:56:35.455839 9108 ts_tablet_manager.cc:1905] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:35.463865 9108 ts_tablet_manager.cc:1918] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:35.464143 9108 log.cc:1196] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/d3bb2dd0b0c54c32b96e19f5bd206aad
I20260812 16:56:35.464668 9108 ts_tablet_manager.cc:1939] T d3bb2dd0b0c54c32b96e19f5bd206aad P 8fe9dc22c1d041ffa92731207ba5ec49: Deleting consensus metadata
I20260812 16:56:35.465116 8991 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 16:56:35.465397 8991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260812 16:56:35.465564 8991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:35.465672 8991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:35.502285 8991 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:37517
I20260812 16:56:35.502384 9316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:37517 every 8 connection(s)
I20260812 16:56:35.503683 8991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:56:35.506860 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 8991
I20260812 16:56:35.507722 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:40923
--local_ip_for_outbound_sockets=127.8.195.130
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=33753
--webserver_interface=127.8.195.130
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:56:35.509922 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:35.507469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:43479)
0812 16:56:35.508419 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:35.508720 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":430,"threads_started":1}
W20260812 16:56:35.510593 9320 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:37423 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:37423: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
I20260812 16:56:35.511540 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:35.511171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:34309)
0812 16:56:35.511343 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:35.511429 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 16:56:35.512414 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:35.512043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:46459)
0812 16:56:35.512192 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:35.512284 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 16:56:35.512681 9320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:56:35.877799 9325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:35.878105 9325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:35.878211 9325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:35.885754 9325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:35.885875 9325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:35.885988 9325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:56:35.896034 9325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:40923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=33753
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:35.897357 9325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:35.898608 9325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:35.912425 9335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:35.912425 9337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:35.913131 9339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:35.914539 9325 server_base.cc:1034] running on GCE node
I20260812 16:56:35.915123 9325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:35.916035 9325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:35.917289 9325 hybrid_clock.cc:648] HybridClock initialized: now 1786553795917225 us; error 40 us; skew 500 ppm
I20260812 16:56:35.917573 9325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:35 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:35.920867 9325 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:56:35.922516 9325 webserver.cc:460] Webserver started at http://127.8.195.130:33753/ using document root <none> and password file <none>
I20260812 16:56:35.923228 9325 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:35.923334 9325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:35.928730 9325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:56:35.931463 9352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:35.932811 9325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 16:56:35.932997 9325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "572c3d3778284c0ea3251f5e58b1eac8"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "d9274b48d4b4aa2ed66ac35679d77bea"
server_key_iv: "7a5eaa363b4100090a9e39a757787f19"
server_key_version: "encryptionkey@0"
I20260812 16:56:35.934118 9325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:35.955084 9325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:35.959533 9325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:35.959842 9325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:35.960929 9325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:35.962383 9325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:35.962524 9325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:35.962677 9325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:35.962769 9325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:35.998364 9325 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40923
I20260812 16:56:35.998459 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40923 every 8 connection(s)
I20260812 16:56:35.999889 9325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:56:36.001688 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9325
I20260812 16:56:36.002327 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:41759
--local_ip_for_outbound_sockets=127.8.195.131
--tserver_master_addrs=127.8.195.190:37423
--webserver_port=45865
--webserver_interface=127.8.195.131
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:56:36.011197 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.003074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:34953)
0812 16:56:36.009585 (+ 6511us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.009918 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6157,"thread_start_us":5919,"threads_started":1}
W20260812 16:56:36.012009 9606 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:37423 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:37423: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
I20260812 16:56:36.013075 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.012629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:39213)
0812 16:56:36.012823 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.012919 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 16:56:36.014057 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.013694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:56735)
0812 16:56:36.013862 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.013951 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 16:56:36.014405 9606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 16:56:36.348878 9616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:36.349249 9616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:36.349436 9616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:36.356904 9616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:36.357019 9616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:36.357267 9616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:56:36.367244 9616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=45865
--tserver_master_addrs=127.8.195.190:37423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:36.368613 9616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:36.369994 9616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:36.383544 9627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:36.383625 9624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:36.383628 9623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:36.384625 9616 server_base.cc:1034] running on GCE node
I20260812 16:56:36.385406 9616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:36.386452 9616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:36.387737 9616 hybrid_clock.cc:648] HybridClock initialized: now 1786553796387695 us; error 51 us; skew 500 ppm
I20260812 16:56:36.388142 9616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:36 dist-test-slave-z8x9 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:36.391718 9616 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:56:36.393378 9616 webserver.cc:460] Webserver started at http://127.8.195.131:45865/ using document root <none> and password file <none>
I20260812 16:56:36.394071 9616 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:36.394181 9616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:36.399817 9616 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 16:56:36.403028 9639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:36.404456 9616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:56:36.404603 9616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "f2562853f325488b9035a1f3caca5a53"
format_stamp: "Formatted at 2026-08-12 16:55:49 on dist-test-slave-z8x9"
server_key: "fe6786ac79dd9eef8804cb1187e2199a"
server_key_iv: "967bdfcd413d56563f9a3fe89b7ba53d"
server_key_version: "encryptionkey@0"
I20260812 16:56:36.405666 9616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:36.429901 9616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:36.434006 9616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:36.434238 9616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:36.435168 9616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:36.436509 9616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:36.436599 9616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:36.436682 9616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:36.436738 9616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:36.471076 9616 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:41759
I20260812 16:56:36.471182 9889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:41759 every 8 connection(s)
I20260812 16:56:36.472586 9616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:56:36.474248 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 9616
I20260812 16:56:36.478067 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.475927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:34947)
0812 16:56:36.476813 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.477109 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":372,"threads_started":1}
W20260812 16:56:36.478879 9892 heartbeater.cc:644] Failed to heartbeat to 127.8.195.190:37423 (0 consecutive failures): Network error: Failed to ping master at 127.8.195.190:37423: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
I20260812 16:56:36.479781 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.479461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:54715)
0812 16:56:36.479605 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.479675 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 16:56:36.480614 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.480276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:37487)
0812 16:56:36.480423 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.480498 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:56:36.480866 9892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 16:56:36.514072 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:36.513264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:60441)
0812 16:56:36.513715 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:36.513848 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":214,"threads_started":1}
I20260812 16:56:37.016098 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:37.015128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:48371)
0812 16:56:37.015675 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:37.015797 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":265,"threads_started":1}
I20260812 16:56:37.482525 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:37.481656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:37045)
0812 16:56:37.482146 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:37.482253 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":237,"threads_started":1}
I20260812 16:56:37.515753 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:37.515029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:45455)
0812 16:56:37.515492 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:37.515578 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":265,"threads_started":1}
I20260812 16:56:38.017889 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:38.017081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:44751)
0812 16:56:38.017623 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:38.017710 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":311,"threads_started":1}
I20260812 16:56:38.484447 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:38.483555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:42239)
0812 16:56:38.484142 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:38.484229 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":303,"threads_started":1}
I20260812 16:56:38.517555 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:38.516698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:49241)
0812 16:56:38.517265 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:38.517368 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260812 16:56:39.019560 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:39.018823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:33783)
0812 16:56:39.019259 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:39.019366 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":227,"threads_started":1}
I20260812 16:56:39.486466 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:39.485588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:57057)
0812 16:56:39.486115 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:39.486234 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":262,"threads_started":1}
I20260812 16:56:39.519184 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:39.518479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:56155)
0812 16:56:39.518941 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:39.519021 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":240,"threads_started":1}
I20260812 16:56:40.021303 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:40.020501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:33071)
0812 16:56:40.020963 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:40.021051 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":231,"threads_started":1}
I20260812 16:56:40.488494 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:40.487494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:33471)
0812 16:56:40.488109 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:40.488248 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":331,"threads_started":1}
I20260812 16:56:40.521004 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:40.520185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:49571)
0812 16:56:40.520731 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:40.520820 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":312,"threads_started":1}
I20260812 16:56:41.023036 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:41.022291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:46363)
0812 16:56:41.022769 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:41.022858 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":225,"threads_started":1}
I20260812 16:56:41.490342 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:41.489579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:37233)
0812 16:56:41.490054 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:41.490136 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
I20260812 16:56:41.522749 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:41.521973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:41265)
0812 16:56:41.522468 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:41.522561 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":290,"threads_started":1}
I20260812 16:56:42.024880 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:42.024040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:40133)
0812 16:56:42.024533 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:42.024641 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":245,"threads_started":1}
I20260812 16:56:42.491958 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:42.491228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:35643)
0812 16:56:42.491673 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:42.491771 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260812 16:56:42.524487 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:42.523685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:51223)
0812 16:56:42.524180 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:42.524265 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":230,"threads_started":1}
I20260812 16:56:43.026763 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:43.025900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:42103)
0812 16:56:43.026458 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:43.026546 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":236,"threads_started":1}
I20260812 16:56:43.493909 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:43.493016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:46107)
0812 16:56:43.493589 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:43.493683 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":288,"threads_started":1}
I20260812 16:56:43.526471 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:43.525605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:36031)
0812 16:56:43.526089 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:43.526186 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260812 16:56:44.028525 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:44.027753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:55983)
0812 16:56:44.028247 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:44.028352 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260812 16:56:44.495627 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:44.494849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:38887)
0812 16:56:44.495361 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:44.495449 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
I20260812 16:56:44.528254 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:44.527446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:46387)
0812 16:56:44.527969 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:44.528076 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":286,"threads_started":1}
I20260812 16:56:45.030689 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:45.029660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:57683)
0812 16:56:45.030316 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:45.030421 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":345,"threads_started":1}
I20260812 16:56:45.497381 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:45.496535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:52649)
0812 16:56:45.497059 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:45.497159 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":262,"threads_started":1}
I20260812 16:56:45.530126 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:45.529372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:49203)
0812 16:56:45.529858 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:45.529943 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":261,"threads_started":1}
I20260812 16:56:46.032568 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:46.031808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:41371)
0812 16:56:46.032305 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:46.032393 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
I20260812 16:56:46.499058 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:46.498342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:37105)
0812 16:56:46.498799 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:46.498883 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":251,"threads_started":1}
I20260812 16:56:46.531919 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:46.531047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:38689)
0812 16:56:46.531609 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:46.531703 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":348,"threads_started":1}
I20260812 16:56:47.034266 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:47.033540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:45311)
0812 16:56:47.033977 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:47.034074 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
I20260812 16:56:47.500927 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:47.500132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:40993)
0812 16:56:47.500634 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:47.500737 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":275,"threads_started":1}
I20260812 16:56:47.533737 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:47.532989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:38917)
0812 16:56:47.533494 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:47.533579 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":292,"threads_started":1}
I20260812 16:56:48.035930 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:48.035202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:51605)
0812 16:56:48.035666 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:48.035759 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
I20260812 16:56:48.502774 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:48.501988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:36533)
0812 16:56:48.502519 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:48.502606 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":318,"threads_started":1}
I20260812 16:56:48.535637 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:48.534841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:34065)
0812 16:56:48.535326 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:48.535437 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":239,"threads_started":1}
I20260812 16:56:49.037786 9969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:49.036869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:41067)
0812 16:56:49.037439 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:49.037584 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":245,"threads_started":1}
I20260812 16:56:49.504676 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:49.503779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:53299)
0812 16:56:49.504379 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:49.504485 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":311,"threads_started":1}
I20260812 16:56:49.537477 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:49.536626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:37027)
0812 16:56:49.537114 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:49.537280 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":260,"threads_started":1}
I20260812 16:56:50.039616 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:50.038812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:55989)
0812 16:56:50.039339 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:50.039428 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":241,"threads_started":1}
I20260812 16:56:50.506916 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:50.505975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:50765)
0812 16:56:50.506599 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:50.506706 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":357,"threads_started":1}
I20260812 16:56:50.539580 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:50.538765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:51963)
0812 16:56:50.539267 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:50.539380 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":271,"threads_started":1}
I20260812 16:56:51.041504 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.040716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.130:46751)
0812 16:56:51.041140 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:51.041314 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260812 16:56:51.481339 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 8991
I20260812 16:56:51.506891 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.482632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.8.195.129:37517)
0812 16:56:51.483512 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:51.483528 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:51.483577 (+ 49us) server_negotiation.cc:407] Connection header received
0812 16:56:51.485416 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:51.485428 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:51.485900 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:51.486312 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:51.488124 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.489630 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.491070 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.491819 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.492052 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:51.499925 (+ 7873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:51.499934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:51.499953 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:51.499999 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:51.503176 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.504022 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.504030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.504037 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.504243 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.504645 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.504651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.504654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.505340 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:51.505670 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:51.506153 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 16:56:51.506604 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":494,"threads_started":1}
I20260812 16:56:51.508564 9310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:51.508903 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.508059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.131:58269)
0812 16:56:51.508569 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:51.508723 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":286,"threads_started":1}
I20260812 16:56:51.595477 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 8991
I20260812 16:56:51.595518 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.594321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:37423 (local address 127.8.195.129:43537)
0812 16:56:51.594957 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:51.595186 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.190:37423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":307,"threads_started":1}
I20260812 16:56:51.621361 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9325
I20260812 16:56:51.639580 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.622315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.8.195.130:40923)
0812 16:56:51.622860 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:51.622875 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:51.622912 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:56:51.624504 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:51.624515 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:51.624949 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:51.625301 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:51.626771 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.628013 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.629383 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.630057 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.630185 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:51.633062 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:51.633070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:51.633085 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:51.633120 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:51.636224 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.636951 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.636957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.636964 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.637149 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.637645 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.637649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.637651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.638239 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:51.638478 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:51.638945 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 16:56:51.639270 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":258,"threads_started":1}
I20260812 16:56:51.641033 9598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:51.725921 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9325
I20260812 16:56:51.759817 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 9616
I20260812 16:56:51.779093 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:51.760883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.8.195.131:41759)
0812 16:56:51.761564 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:51.761578 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:51.761621 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:56:51.763377 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:51.763387 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:51.763807 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:51.764098 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:51.765648 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.766881 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.768123 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:51.768810 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:51.768950 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:51.771813 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:51.771821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:51.771837 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:51.771875 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:51.775449 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.776223 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.776230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.776236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.776429 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:51.776847 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:51.776851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:51.776853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:51.777587 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:51.777833 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:51.778279 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 16:56:51.778742 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":364,"threads_started":1}
I20260812 16:56:51.780568 9876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:51.864637 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 9616
2026-08-12T16:56:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65983 ms)
[----------] 30 tests from SecurityITest (320253 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:56:51 dist-test-slave-z8x9 krb5kdc[10002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:56:51 dist-test-slave-z8x9 krb5kdc[10002](info): set up 2 sockets
Aug 12 16:56:51 dist-test-slave-z8x9 krb5kdc[10002](info): commencing operation
krb5kdc: starting...
W20260812 16:56:53.313866 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:56:53 dist-test-slave-z8x9 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:56:53.638743 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34461
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:53.994786 10030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:53.995211 10030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:53.995381 10030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:54.002856 10030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:56:54.003028 10030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:54.003149 10030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:54.003271 10030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:56:54.003363 10030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:56:54.013515 10030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:54.014926 10030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:54.016274 10030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:54.028898 10041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:54.028985 10045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:54.028971 10048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:54.029597 10030 server_base.cc:1034] running on GCE node
I20260812 16:56:54.030471 10030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:54.031854 10030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:54.033063 10030 hybrid_clock.cc:648] HybridClock initialized: now 1786553814033020 us; error 33 us; skew 500 ppm
I20260812 16:56:54.033449 10030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:54 dist-test-slave-z8x9 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553814, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:54.036609 10030 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:56:54.038216 10030 webserver.cc:460] Webserver started at http://127.8.195.190:40351/ using document root <none> and password file <none>
I20260812 16:56:54.039026 10030 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:54.039181 10030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:54.039505 10030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:54.042699 10030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "784da76db9cc4ec59f70e41ac2ac3c7b"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "9103d4f6108a212952d3d4e2ac84a5d5"
server_key_iv: "6b46275e6dc70726d45c9ef27c8ceed4"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.043514 10030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "784da76db9cc4ec59f70e41ac2ac3c7b"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "9103d4f6108a212952d3d4e2ac84a5d5"
server_key_iv: "6b46275e6dc70726d45c9ef27c8ceed4"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.049381 10030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 16:56:54.053030 10062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:54.054493 10030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:56:54.054704 10030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "784da76db9cc4ec59f70e41ac2ac3c7b"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "9103d4f6108a212952d3d4e2ac84a5d5"
server_key_iv: "6b46275e6dc70726d45c9ef27c8ceed4"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.054939 10030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:54.075582 10030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:54.079939 10030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:54.080260 10030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:54.099676 10030 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34461
I20260812 16:56:54.099710 10180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34461 every 8 connection(s)
I20260812 16:56:54.101152 10030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:56:54.104952 10181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:54.110461 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10030
I20260812 16:56:54.110715 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:56:54.111397 8974 external_mini_cluster.cc:1428] Setting key bb29fedc3aa00b0378f9fec886ae8fff
I20260812 16:56:54.116532 10181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: Bootstrap starting.
I20260812 16:56:54.120548 10181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:54.121963 10181 log.cc:824] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:54.125671 10181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: No bootstrap required, opened a new log
Aug 12 16:56:54 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:54.134208 10181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } }
I20260812 16:56:54.134620 10181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:54.134758 10181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784da76db9cc4ec59f70e41ac2ac3c7b, State: Initialized, Role: FOLLOWER
I20260812 16:56:54.135421 10181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } }
I20260812 16:56:54.135689 10181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:56:54.135854 10181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:56:54.136073 10181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:54.137454 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:54.113536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.8.195.190:34461)
0812 16:56:54.114430 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:54.114442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:54.114487 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:56:54.116239 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:54.116265 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:54.116641 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:54.117029 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:54.119288 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.120613 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.122716 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.123753 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.123898 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:54.129327 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:54.129336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:54.129351 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:54.129398 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:54.132741 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:54.133568 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:54.133574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:54.133581 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:54.133765 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:54.134204 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:54.134209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:54.134211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:54.134920 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:54.135254 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:54.135799 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 16:56:54.136263 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":428,"threads_started":1}
I20260812 16:56:54.138728 10181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } }
I20260812 16:56:54.139305 10181 leader_election.cc:302] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 784da76db9cc4ec59f70e41ac2ac3c7b; no voters:
I20260812 16:56:54.140039 10181 leader_election.cc:288] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:56:54.140185 10193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:56:54.141098 10193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [term 1 LEADER]: Becoming Leader. State: Replica: 784da76db9cc4ec59f70e41ac2ac3c7b, State: Running, Role: LEADER
I20260812 16:56:54.141752 10193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } }
I20260812 16:56:54.142526 10181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:56:54.144891 10197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 784da76db9cc4ec59f70e41ac2ac3c7b. Latest consensus state: current_term: 1 leader_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } } }
I20260812 16:56:54.145431 10197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:54.144876 10195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "784da76db9cc4ec59f70e41ac2ac3c7b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34461 } } }
I20260812 16:56:54.146023 10195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:54.154467 10203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:56:54.159528 10203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:56:54.168241 10203 catalog_manager.cc:1347] Generated new cluster ID: 622d3eb9b87c4e48af237029bd322900
I20260812 16:56:54.168414 10203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:56:54.227384 10203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:56:54.228847 10203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:56:54.236629 10203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: Generated new TSK 0
I20260812 16:56:54.237569 10203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:56:54.325214 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:54.677917 10242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:54.678223 10242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:54.678330 10242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:54.685655 10242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:54.685776 10242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:54.685896 10242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:56:54.695633 10242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:54.696997 10242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:54.698482 10242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:54.711844 10260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:54.711933 10257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:54.711886 10253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:54.714901 10242 server_base.cc:1034] running on GCE node
I20260812 16:56:54.715570 10242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:54.716643 10242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:54.717849 10242 hybrid_clock.cc:648] HybridClock initialized: now 1786553814717804 us; error 35 us; skew 500 ppm
I20260812 16:56:54.718186 10242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:54 dist-test-slave-z8x9 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553814, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:54.721529 10242 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:56:54.723125 10242 webserver.cc:460] Webserver started at http://127.8.195.129:46357/ using document root <none> and password file <none>
I20260812 16:56:54.723865 10242 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:54.724015 10242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:54.724355 10242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:54.727363 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "a5e7110c5c634a83ac38450c84f47bce"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "6487279b81fb9991a54cb479f4774af2"
server_key_iv: "1c970088cfc3eb107d7b77458f096f6e"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.728204 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "a5e7110c5c634a83ac38450c84f47bce"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "6487279b81fb9991a54cb479f4774af2"
server_key_iv: "1c970088cfc3eb107d7b77458f096f6e"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.733347 10242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 16:56:54.736855 10277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:54.738164 10242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:56:54.738353 10242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "a5e7110c5c634a83ac38450c84f47bce"
format_stamp: "Formatted at 2026-08-12 16:56:54 on dist-test-slave-z8x9"
server_key: "6487279b81fb9991a54cb479f4774af2"
server_key_iv: "1c970088cfc3eb107d7b77458f096f6e"
server_key_version: "encryptionkey@0"
I20260812 16:56:54.738564 10242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:54.760548 10242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:54.764367 10242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:54.764612 10242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:54.765574 10242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:54.766824 10242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:54.766902 10242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:54.766987 10242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:54.767067 10242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:54.801507 10242 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:41233
I20260812 16:56:54.801581 10517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:41233 every 8 connection(s)
I20260812 16:56:54.802881 10242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:56:54.808218 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10242
I20260812 16:56:54.808470 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:56:54.809224 8974 external_mini_cluster.cc:1428] Setting key 4ead0db1abd1b3bb8f669e53de5d60d8
Aug 12 16:56:54 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553814, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:54.825850 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:54.805512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49247 (local address 127.8.195.190:34461)
0812 16:56:54.805995 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:54.806000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:54.806815 (+ 815us) server_negotiation.cc:407] Connection header received
0812 16:56:54.808796 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:54.808803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:54.808904 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:54.809024 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:54.811066 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.812169 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.813829 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.814549 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.814694 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:54.818710 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:54.818718 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:54.818725 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:54.818769 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:54.822139 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:54.823103 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:54.823109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:54.823114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:54.823281 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:54.823810 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:54.823817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:54.823822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:54.824182 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:54.824317 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:54.825236 (+ 919us) server_negotiation.cc:299] Negotiation successful
0812 16:56:54.825484 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":266,"threads_started":1}
I20260812 16:56:54.826857 10527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:54.805923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34461 (local address 127.8.195.129:49247)
0812 16:56:54.806636 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:54.806681 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:54.808484 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:54.809309 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:54.809343 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:54.810051 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:54.810828 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:54.810853 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.812339 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.812343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:54.813683 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:54.813696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:54.814755 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:54.814759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:54.814956 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:54.815660 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:54.815682 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:54.818470 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:54.822301 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:54.822312 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:54.822331 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:54.822940 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:54.823416 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:54.823423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:54.823429 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:54.823670 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:54.824385 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:54.824395 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:54.824828 (+ 433us) client_negotiation.cc:769] Sending connection context
0812 16:56:54.825096 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 16:56:54.825645 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":277,"threads_started":1}
I20260812 16:56:54.828609 10521 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34461
I20260812 16:56:54.828930 10521 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:54.829707 10521 heartbeater.cc:511] Master 127.8.195.190:34461 requested a full tablet report, sending...
I20260812 16:56:54.832043 10119 ts_manager.cc:194] Registered new tserver with Master: a5e7110c5c634a83ac38450c84f47bce (127.8.195.129:41233)
I20260812 16:56:54.834324 10119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:49247
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:56:54.880848 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:55.226594 10535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:55.226881 10535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:55.226982 10535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:55.234298 10535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:55.234416 10535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:55.234530 10535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:56:55.244851 10535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:55.246169 10535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:55.247484 10535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:55.261138 10551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:55.261361 10554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:55.261323 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:55.262234 10535 server_base.cc:1034] running on GCE node
I20260812 16:56:55.262880 10535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:55.263854 10535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:55.265093 10535 hybrid_clock.cc:648] HybridClock initialized: now 1786553815265039 us; error 42 us; skew 500 ppm
I20260812 16:56:55.265437 10535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:55 dist-test-slave-z8x9 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:55.268707 10535 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:56:55.270174 10535 webserver.cc:460] Webserver started at http://127.8.195.130:42467/ using document root <none> and password file <none>
I20260812 16:56:55.270870 10535 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:55.271019 10535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:55.271373 10535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:55.274289 10535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "0460bbf0a8b843b4bbce1c61dce7b796"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "b7e41c7544d771d835977b42e75971c4"
server_key_iv: "7ace133763557e3a9efdc1a3496ed236"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.275041 10535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "0460bbf0a8b843b4bbce1c61dce7b796"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "b7e41c7544d771d835977b42e75971c4"
server_key_iv: "7ace133763557e3a9efdc1a3496ed236"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.280098 10535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:56:55.283947 10568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.285379 10535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:56:55.285562 10535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "0460bbf0a8b843b4bbce1c61dce7b796"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "b7e41c7544d771d835977b42e75971c4"
server_key_iv: "7ace133763557e3a9efdc1a3496ed236"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.285791 10535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:55.305421 10535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:55.309710 10535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:55.310046 10535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:55.311070 10535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:55.312505 10535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:55.312613 10535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.312772 10535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:55.312851 10535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.348073 10535 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40547
I20260812 16:56:55.348140 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40547 every 8 connection(s)
I20260812 16:56:55.349591 10535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:56:55.353116 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10535
I20260812 16:56:55.353336 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:56:55.353775 8974 external_mini_cluster.cc:1428] Setting key 9dce365f6efd5bf21fbd5168cd735bee
Aug 12 16:56:55 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:55.373462 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.352267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:52839 (local address 127.8.195.190:34461)
0812 16:56:55.352747 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:55.352754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:55.353987 (+ 1233us) server_negotiation.cc:407] Connection header received
0812 16:56:55.356153 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:55.356159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:55.356247 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:55.356395 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:55.358031 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.359253 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.360919 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.361683 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.361834 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:55.365722 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:55.365732 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:55.365739 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:55.365789 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:55.369809 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.370655 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.370667 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.370669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.370808 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.371300 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.371309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.371315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.371720 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:55.371898 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:55.372643 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 16:56:55.372918 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":214,"threads_started":1}
I20260812 16:56:55.374194 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.352905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34461 (local address 127.8.195.130:52839)
0812 16:56:55.353783 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:55.353841 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:55.355858 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:55.356602 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:55.356610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:55.357241 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:55.357852 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:55.357871 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.359418 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.359422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:55.360762 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:55.360775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.361898 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.361903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:55.362105 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:55.362855 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:55.362883 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:55.365452 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:55.369978 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:55.369987 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:55.370003 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:55.370498 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:55.370932 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:55.370937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:55.370947 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:55.371145 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:55.371881 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:55.371888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:55.372237 (+ 349us) client_negotiation.cc:769] Sending connection context
0812 16:56:55.372494 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 16:56:55.372919 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":384,"threads_started":1}
I20260812 16:56:55.375746 10790 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34461
I20260812 16:56:55.376080 10790 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:55.376844 10790 heartbeater.cc:511] Master 127.8.195.190:34461 requested a full tablet report, sending...
I20260812 16:56:55.378508 10119 ts_manager.cc:194] Registered new tserver with Master: 0460bbf0a8b843b4bbce1c61dce7b796 (127.8.195.130:40547)
I20260812 16:56:55.379379 10119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:52839
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:56:55.426935 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:56:55.768877 10796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:55.769265 10796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:55.769429 10796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:55.776651 10796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:55.776809 10796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:55.776999 10796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:56:55.786733 10796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:55.788066 10796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:55.789620 10796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:55.803117 10809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:55.803544 10812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:55.803092 10810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:55.805269 10796 server_base.cc:1034] running on GCE node
I20260812 16:56:55.805892 10796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:55.806865 10796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:55.808115 10796 hybrid_clock.cc:648] HybridClock initialized: now 1786553815808049 us; error 48 us; skew 500 ppm
I20260812 16:56:55.808444 10796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:56:55 dist-test-slave-z8x9 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:56:55.815172 10796 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:56:55.816773 10796 webserver.cc:460] Webserver started at http://127.8.195.131:33181/ using document root <none> and password file <none>
I20260812 16:56:55.817600 10796 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:55.817751 10796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:55.818078 10796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:55.820979 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "ca4209e8c820478cb2b0f6a5f473f9d5"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "09e4df598b8054ecd24fdf7a36b446ac"
server_key_iv: "a18f806836ad933bea312be02fde212c"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.821838 10796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "ca4209e8c820478cb2b0f6a5f473f9d5"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "09e4df598b8054ecd24fdf7a36b446ac"
server_key_iv: "a18f806836ad933bea312be02fde212c"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.827232 10796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 16:56:55.830624 10825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.831861 10796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:56:55.832041 10796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "ca4209e8c820478cb2b0f6a5f473f9d5"
format_stamp: "Formatted at 2026-08-12 16:56:55 on dist-test-slave-z8x9"
server_key: "09e4df598b8054ecd24fdf7a36b446ac"
server_key_iv: "a18f806836ad933bea312be02fde212c"
server_key_version: "encryptionkey@0"
I20260812 16:56:55.832245 10796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:55.837461 10521 heartbeater.cc:503] Master 127.8.195.190:34461 was elected leader, sending a full tablet report...
I20260812 16:56:55.854784 10796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:55.860574 10796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:55.860888 10796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:55.861968 10796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:55.863436 10796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:55.863548 10796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.863686 10796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:55.863755 10796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:55.916571 10796 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:35191
I20260812 16:56:55.916669 11036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:35191 every 8 connection(s)
I20260812 16:56:55.918201 10796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:56:55.920684 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 10796
I20260812 16:56:55.920890 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:56:55.921527 8974 external_mini_cluster.cc:1428] Setting key 23cef573a1aa7ec6f865f5501c9e6c86
Aug 12 16:56:55 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:56:55.943115 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.922765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:38897 (local address 127.8.195.190:34461)
0812 16:56:55.923697 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:55.923705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:55.923729 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:56:55.926570 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:55.926575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:55.926646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:55.926739 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:55.929317 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.930633 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.932400 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.932831 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.932938 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:55.937068 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:55.937075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:55.937079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:55.937120 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:55.939924 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.940596 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.940600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.940602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.940722 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.941260 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.941264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.941266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.941609 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:55.941739 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:55.942463 (+ 724us) server_negotiation.cc:299] Negotiation successful
0812 16:56:55.942730 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":987,"threads_started":1}
I20260812 16:56:55.944056 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.922162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34461 (local address 127.8.195.131:38897)
0812 16:56:55.923399 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:55.923447 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:55.926099 (+ 2652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:55.927067 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:55.927082 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:55.928035 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:55.929003 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:55.929037 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.930857 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.930867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:55.932239 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:55.932252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.933076 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.933081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:55.933391 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:55.934145 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:56:55.934168 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:55.936781 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:55.940053 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:55.940060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:55.940072 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:55.940493 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:55.940895 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:56:55.940898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:56:55.940901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:56:55.941080 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:56:55.941717 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:55.941723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:55.942037 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 16:56:55.942269 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 16:56:55.942727 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":650,"threads_started":1}
I20260812 16:56:55.945554 11037 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34461
I20260812 16:56:55.945864 11037 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:55.946580 11037 heartbeater.cc:511] Master 127.8.195.190:34461 requested a full tablet report, sending...
I20260812 16:56:55.948007 10119 ts_manager.cc:194] Registered new tserver with Master: ca4209e8c820478cb2b0f6a5f473f9d5 (127.8.195.131:35191)
I20260812 16:56:55.949101 10119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:38897
I20260812 16:56:55.950448 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:56:55.968323 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.955078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.8.195.190:34461)
0812 16:56:55.955236 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:55.955241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:55.955706 (+ 465us) server_negotiation.cc:407] Connection header received
0812 16:56:55.955882 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:55.955887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:55.955959 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:55.956049 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:55.957648 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.958476 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.959931 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:55.960327 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:55.960419 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:55.962723 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:55.962727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:55.962730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:55.962762 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:55.965557 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.966251 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.966258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.966261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.966402 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:55.966796 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:55.966803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:55.966808 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:55.967139 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:55.967406 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:55.967757 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:56:55.967996 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:56:55.972527 10119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:56:55.979660 10119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:56:56.018258 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.992413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40547 (local address 127.0.0.1:37616)
0812 16:56:56.004522 (+ 12109us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.004584 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.004835 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.010125 (+ 5290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.010137 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.010184 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.010680 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.010692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.012834 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.012840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.015471 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.015485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.016980 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.016987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.017271 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.017326 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.017488 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.017760 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11734,"mutex_wait_us":25,"thread_start_us":11542,"threads_started":1}
I20260812 16:56:56.019392 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.992943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.8.195.130:40547)
0812 16:56:56.009308 (+ 16365us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.009321 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.009359 (+ 38us) server_negotiation.cc:407] Connection header received
0812 16:56:56.009481 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.009493 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.009829 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.010074 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.010915 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.012647 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.015712 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.016748 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.016905 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.017243 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.018763 (+ 1520us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.018967 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16112,"thread_start_us":17354,"threads_started":1}
I20260812 16:56:56.019762 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.992495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41233 (local address 127.0.0.1:41370)
0812 16:56:56.011594 (+ 19099us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.011620 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.011770 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.012515 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.012519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.012547 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.013099 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.013113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.014843 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.014851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.016951 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.016967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.018896 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.018905 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.019087 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.019122 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.019223 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.019354 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18939,"mutex_wait_us":59,"thread_start_us":20130,"threads_started":1}
I20260812 16:56:56.019819 11064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.992521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.8.195.129:41233)
0812 16:56:56.010973 (+ 18452us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.010984 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.011926 (+ 942us) server_negotiation.cc:407] Connection header received
0812 16:56:56.012037 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.012047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.012288 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.013273 (+ 985us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.013579 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.014690 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.017195 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.018669 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.018845 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.019067 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.019289 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.019466 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18266,"thread_start_us":18493,"threads_started":1}
I20260812 16:56:56.021595 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.992885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:35191 (local address 127.0.0.1:51706)
0812 16:56:56.011885 (+ 19000us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.011904 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.012008 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.012958 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.012965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.013001 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.013686 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.013700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.015656 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.015661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.017873 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.017890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.020719 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.020725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.020921 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.020951 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.021052 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.021206 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18873,"mutex_wait_us":5,"thread_start_us":18356,"threads_started":1}
I20260812 16:56:56.021641 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:55.993496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51706 (local address 127.8.195.131:35191)
0812 16:56:56.011038 (+ 17542us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.011050 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.012125 (+ 1075us) server_negotiation.cc:407] Connection header received
0812 16:56:56.012228 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.012234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.012455 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.012648 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.013899 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.015484 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.018061 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.020489 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.020625 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.020851 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.021118 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.021308 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17341,"thread_start_us":17176,"threads_started":1}
I20260812 16:56:56.023671 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 (DEFAULT_TABLE table=test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:56.023671 10372 tablet_service.cc:1467] Processing CreateTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 (DEFAULT_TABLE table=test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:56.024499 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 (DEFAULT_TABLE table=test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:56.025236 10372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218f0f0a34a14e3ab4bda99192826bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:56.025267 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218f0f0a34a14e3ab4bda99192826bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:56.025887 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218f0f0a34a14e3ab4bda99192826bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:56.037925 11079 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Bootstrap starting.
I20260812 16:56:56.039889 11081 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Bootstrap starting.
I20260812 16:56:56.041941 11082 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Bootstrap starting.
I20260812 16:56:56.042905 11079 tablet_bootstrap.cc:654] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:56.043598 11081 tablet_bootstrap.cc:654] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:56.044240 11079 log.cc:824] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:56.044837 11081 log.cc:824] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:56.045480 11082 tablet_bootstrap.cc:654] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:56.046516 11082 log.cc:824] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:56.049780 11079 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: No bootstrap required, opened a new log
I20260812 16:56:56.050057 11079 ts_tablet_manager.cc:1397] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 16:56:56.050158 11082 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: No bootstrap required, opened a new log
I20260812 16:56:56.050468 11082 ts_tablet_manager.cc:1397] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 16:56:56.054100 11081 tablet_bootstrap.cc:492] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: No bootstrap required, opened a new log
I20260812 16:56:56.054378 11081 ts_tablet_manager.cc:1397] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260812 16:56:56.058454 11079 raft_consensus.cc:348] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.058988 11079 raft_consensus.cc:374] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:56.059093 11079 raft_consensus.cc:729] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4209e8c820478cb2b0f6a5f473f9d5, State: Initialized, Role: FOLLOWER
I20260812 16:56:56.059706 11079 consensus_queue.cc:260] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.060920 11037 heartbeater.cc:503] Master 127.8.195.190:34461 was elected leader, sending a full tablet report...
I20260812 16:56:56.061728 11079 ts_tablet_manager.cc:1428] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 16:56:56.061676 11082 raft_consensus.cc:348] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.062124 11082 raft_consensus.cc:374] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:56.062216 11082 raft_consensus.cc:729] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e7110c5c634a83ac38450c84f47bce, State: Initialized, Role: FOLLOWER
I20260812 16:56:56.062819 11082 consensus_queue.cc:260] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.064606 11082 ts_tablet_manager.cc:1428] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 16:56:56.065042 11081 raft_consensus.cc:348] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.065531 11081 raft_consensus.cc:374] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:56.065642 11081 raft_consensus.cc:729] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0460bbf0a8b843b4bbce1c61dce7b796, State: Initialized, Role: FOLLOWER
I20260812 16:56:56.066215 11081 consensus_queue.cc:260] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.067462 10790 heartbeater.cc:503] Master 127.8.195.190:34461 was elected leader, sending a full tablet report...
I20260812 16:56:56.068004 11081 ts_tablet_manager.cc:1428] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260812 16:56:56.101761 10791 tablet.cc:2369] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:56:56.170109 11038 tablet.cc:2369] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 16:56:56.305785 10523 tablet.cc:2369] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:56.389127 11088 raft_consensus.cc:482] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:56:56.389499 11088 raft_consensus.cc:504] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.391515 11088 leader_election.cc:288] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e7110c5c634a83ac38450c84f47bce (127.8.195.129:41233), 0460bbf0a8b843b4bbce1c61dce7b796 (127.8.195.130:40547)
I20260812 16:56:56.398568 11064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.392016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:58477 (local address 127.8.195.129:41233)
0812 16:56:56.392241 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.392250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.392278 (+ 28us) server_negotiation.cc:407] Connection header received
0812 16:56:56.392376 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.392385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.392468 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.392583 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.393043 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.394113 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.396501 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.397657 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.397833 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.397912 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.398153 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.398307 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:56:56.398581 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.391729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41233 (local address 127.8.195.131:58477)
0812 16:56:56.391952 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.391974 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.392127 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.392623 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.392628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.392664 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.392881 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.392896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.394268 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.394272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.396328 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.396344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.397823 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.397828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.397968 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.397997 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.398092 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.398200 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":1}
I20260812 16:56:56.399464 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.392164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:44165 (local address 127.8.195.130:40547)
0812 16:56:56.392388 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.392395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.392830 (+ 435us) server_negotiation.cc:407] Connection header received
0812 16:56:56.393058 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.393064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.393153 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.393289 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.393786 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.394654 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.396909 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.398184 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.398343 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.398410 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.399126 (+ 716us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.399270 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 16:56:56.399513 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.392015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40547 (local address 127.8.195.131:44165)
0812 16:56:56.392673 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.392690 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.392853 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.393326 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.393329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.393351 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.393621 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.393628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.394809 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.394814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.396762 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.396772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.398771 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.398779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.398962 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.398985 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.399060 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.399148 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":343,"threads_started":1}
I20260812 16:56:56.399926 11089 raft_consensus.cc:482] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:56:56.399915 10417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7110c5c634a83ac38450c84f47bce" is_pre_election: true
I20260812 16:56:56.400399 11089 raft_consensus.cc:504] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.400688 10695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" is_pre_election: true
I20260812 16:56:56.401252 10695 raft_consensus.cc:2454] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4209e8c820478cb2b0f6a5f473f9d5 in term 0.
I20260812 16:56:56.401329 10417 raft_consensus.cc:2454] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4209e8c820478cb2b0f6a5f473f9d5 in term 0.
I20260812 16:56:56.402025 10830 leader_election.cc:302] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0460bbf0a8b843b4bbce1c61dce7b796, ca4209e8c820478cb2b0f6a5f473f9d5; no voters:
I20260812 16:56:56.402453 11088 raft_consensus.cc:2781] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:56:56.402617 11088 raft_consensus.cc:482] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:56:56.402585 11089 leader_election.cc:288] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0460bbf0a8b843b4bbce1c61dce7b796 (127.8.195.130:40547), ca4209e8c820478cb2b0f6a5f473f9d5 (127.8.195.131:35191)
I20260812 16:56:56.402767 11088 raft_consensus.cc:3037] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:56.405660 11088 raft_consensus.cc:504] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.406450 11088 leader_election.cc:288] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [CANDIDATE]: Term 1 election: Requested vote from peers a5e7110c5c634a83ac38450c84f47bce (127.8.195.129:41233), 0460bbf0a8b843b4bbce1c61dce7b796 (127.8.195.130:40547)
I20260812 16:56:56.407119 10695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0460bbf0a8b843b4bbce1c61dce7b796"
I20260812 16:56:56.407332 10695 raft_consensus.cc:3037] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:56.410161 10695 raft_consensus.cc:2454] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4209e8c820478cb2b0f6a5f473f9d5 in term 1.
I20260812 16:56:56.410578 10830 leader_election.cc:302] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0460bbf0a8b843b4bbce1c61dce7b796, ca4209e8c820478cb2b0f6a5f473f9d5; no voters:
I20260812 16:56:56.410884 11088 raft_consensus.cc:2781] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:56:56.411450 11088 raft_consensus.cc:686] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 1 LEADER]: Becoming Leader. State: Replica: ca4209e8c820478cb2b0f6a5f473f9d5, State: Running, Role: LEADER
I20260812 16:56:56.411978 11088 consensus_queue.cc:237] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } }
I20260812 16:56:56.414034 10417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e7110c5c634a83ac38450c84f47bce"
I20260812 16:56:56.414330 10417 raft_consensus.cc:3037] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:56.415616 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.402923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40547 (local address 127.8.195.129:36355)
0812 16:56:56.405721 (+ 2798us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.405744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.405941 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.406613 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.406621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.406656 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.407167 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.407183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.408760 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.408769 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.412369 (+ 3600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.412393 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.414288 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.414305 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.414669 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.414732 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.414861 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.415051 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"thread_start_us":2446,"threads_started":1}
I20260812 16:56:56.416543 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.403838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:36355 (local address 127.8.195.130:40547)
0812 16:56:56.404169 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.404182 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.405828 (+ 1646us) server_negotiation.cc:407] Connection header received
0812 16:56:56.406160 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.406169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.406275 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.406399 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.407391 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.408608 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.412555 (+ 3947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.413781 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.413957 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.414026 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.415956 (+ 1930us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.416169 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":205}
I20260812 16:56:56.418054 10417 raft_consensus.cc:2454] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4209e8c820478cb2b0f6a5f473f9d5 in term 1.
I20260812 16:56:56.418890 10695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "a5e7110c5c634a83ac38450c84f47bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" is_pre_election: true
I20260812 16:56:56.419229 10695 raft_consensus.cc:2379] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5e7110c5c634a83ac38450c84f47bce in current term 1: Already voted for candidate ca4209e8c820478cb2b0f6a5f473f9d5 in this term.
I20260812 16:56:56.419555 10111 catalog_manager.cc:5495] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca4209e8c820478cb2b0f6a5f473f9d5 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 35191 } health_report { overall_health: HEALTHY } } }
I20260812 16:56:56.422458 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.403808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:47277 (local address 127.8.195.131:35191)
0812 16:56:56.404017 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.404023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.413320 (+ 9297us) server_negotiation.cc:407] Connection header received
0812 16:56:56.413515 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.413520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.413590 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.413680 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:56.414610 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.415756 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.420382 (+ 4626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.421586 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.421744 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.421793 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.422196 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.422287 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:56:56.422590 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.403200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:35191 (local address 127.8.195.129:47277)
0812 16:56:56.413229 (+ 10029us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:56.413248 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:56.413382 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:56.413840 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:56.413847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:56.413872 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:56.414387 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.414401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.415943 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.415953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.420212 (+ 4259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:56.420225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.421825 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.421832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:56.421991 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.422024 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 16:56:56.422133 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 16:56:56.422258 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9806,"thread_start_us":10265,"threads_started":1}
I20260812 16:56:56.423540 10945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218f0f0a34a14e3ab4bda99192826bc1" candidate_uuid: "a5e7110c5c634a83ac38450c84f47bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4209e8c820478cb2b0f6a5f473f9d5" is_pre_election: true
I20260812 16:56:56.424686 10279 leader_election.cc:302] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5e7110c5c634a83ac38450c84f47bce; no voters: 0460bbf0a8b843b4bbce1c61dce7b796, ca4209e8c820478cb2b0f6a5f473f9d5
I20260812 16:56:56.425086 11089 raft_consensus.cc:2726] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 16:56:56.463606 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.457688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.8.195.131:35191)
0812 16:56:56.457868 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.457876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.457959 (+ 83us) server_negotiation.cc:407] Connection header received
0812 16:56:56.458215 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.458220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.458326 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.458526 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:56:56.458940 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.459769 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.461400 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.461819 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.461963 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.462474 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:56:56.462983 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:56:56.463145 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.463204 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.463300 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:56:56.471774 10695 raft_consensus.cc:1261] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 1 FOLLOWER]: Refusing update from remote peer ca4209e8c820478cb2b0f6a5f473f9d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:56.471813 10417 raft_consensus.cc:1261] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 1 FOLLOWER]: Refusing update from remote peer ca4209e8c820478cb2b0f6a5f473f9d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:56.472662 11099 consensus_queue.cc:1035] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0460bbf0a8b843b4bbce1c61dce7b796" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:56:56.472981 11088 consensus_queue.cc:1035] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e7110c5c634a83ac38450c84f47bce" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:56:56.485348 11102 mvcc.cc:205] Tried to move back new op lower bound from 7317724432258347008 to 7317724432028332032. Current Snapshot: MvccSnapshot[applied={T|T < 7317724432258347008}]
I20260812 16:56:56.503505 10119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:56.503823 10119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:56.508003 10119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 784da76db9cc4ec59f70e41ac2ac3c7b: Sending DeleteTablet for 3 replicas of tablet 218f0f0a34a14e3ab4bda99192826bc1
I20260812 16:56:56.509384 10910 tablet_service.cc:1514] Processing DeleteTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:56 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:51706
I20260812 16:56:56.509346 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:56 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:37616
I20260812 16:56:56.509382 10372 tablet_service.cc:1514] Processing DeleteTablet for tablet 218f0f0a34a14e3ab4bda99192826bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:56 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:41370
I20260812 16:56:56.515151 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10242
I20260812 16:56:56.520745 11132 tablet_replica.cc:330] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: stopping tablet replica
I20260812 16:56:56.521521 11132 raft_consensus.cc:2229] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:56.522325 11132 raft_consensus.cc:2258] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:56.522753 11129 tablet_replica.cc:330] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: stopping tablet replica
I20260812 16:56:56.523198 11129 raft_consensus.cc:2229] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:56:56.523978 11129 raft_consensus.cc:2258] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:56.525283 11132 ts_tablet_manager.cc:1905] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:56.525997 11129 ts_tablet_manager.cc:1905] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:56.526674 11124 tablet_replica.cc:330] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: stopping tablet replica
I20260812 16:56:56.533416 11132 ts_tablet_manager.cc:1918] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:56.533566 11129 ts_tablet_manager.cc:1918] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:56.533758 11129 log.cc:1196] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/218f0f0a34a14e3ab4bda99192826bc1
I20260812 16:56:56.534231 11124 raft_consensus.cc:2229] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:56.534379 11129 ts_tablet_manager.cc:1939] T 218f0f0a34a14e3ab4bda99192826bc1 P ca4209e8c820478cb2b0f6a5f473f9d5: Deleting consensus metadata
I20260812 16:56:56.534937 11124 raft_consensus.cc:2258] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:56.535486 11132 log.cc:1196] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/218f0f0a34a14e3ab4bda99192826bc1
I20260812 16:56:56.536158 11132 ts_tablet_manager.cc:1939] T 218f0f0a34a14e3ab4bda99192826bc1 P 0460bbf0a8b843b4bbce1c61dce7b796: Deleting consensus metadata
I20260812 16:56:56.537644 11124 ts_tablet_manager.cc:1905] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:56.538267 10071 catalog_manager.cc:4841] TS 0460bbf0a8b843b4bbce1c61dce7b796 (127.8.195.130:40547): tablet 218f0f0a34a14e3ab4bda99192826bc1 (table test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]) successfully deleted
I20260812 16:56:56.540055 10067 catalog_manager.cc:4841] TS ca4209e8c820478cb2b0f6a5f473f9d5 (127.8.195.131:35191): tablet 218f0f0a34a14e3ab4bda99192826bc1 (table test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]) successfully deleted
I20260812 16:56:56.543756 11124 ts_tablet_manager.cc:1918] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:56.543937 11124 log.cc:1196] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/218f0f0a34a14e3ab4bda99192826bc1
I20260812 16:56:56.544569 11124 ts_tablet_manager.cc:1939] T 218f0f0a34a14e3ab4bda99192826bc1 P a5e7110c5c634a83ac38450c84f47bce: Deleting consensus metadata
I20260812 16:56:56.546612 10069 catalog_manager.cc:4841] TS a5e7110c5c634a83ac38450c84f47bce (127.8.195.129:41233): tablet 218f0f0a34a14e3ab4bda99192826bc1 (table test-table [id=3830ed746ff64e218ea64d5ecd6aac8e]) successfully deleted
Aug 12 16:56:56 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:56:56.556653 11064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.517331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.8.195.129:41233)
0812 16:56:56.517524 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.517530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.531065 (+ 13535us) server_negotiation.cc:407] Connection header received
0812 16:56:56.531600 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.531607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.531700 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.531777 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:56.533847 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.535086 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.537350 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.538056 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.538181 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.549099 (+ 10918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:56.549115 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:56.549135 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:56.549212 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:56.553424 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.554267 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.554277 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.554289 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.554459 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.554862 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.554867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.554870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.555354 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:56.555673 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.555994 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.556238 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:56:56.557452 10499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:56.653870 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10242
I20260812 16:56:56.688148 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10535
Aug 12 16:56:56 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:56:56.706311 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.688990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.8.195.130:40547)
0812 16:56:56.689235 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.689241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.689262 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:56:56.689517 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.689523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.689617 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.689737 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:56.691508 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.692226 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.694150 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.694579 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.694677 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.700169 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:56.700184 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:56.700200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:56.700236 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:56.703401 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.704214 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.704222 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.704224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.704372 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.704761 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.704764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.704766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.705120 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:56.705450 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.705775 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.705948 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:56:56.707024 10784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:56.796998 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10535
I20260812 16:56:56.835265 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10796
Aug 12 16:56:56 dist-test-slave-z8x9 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:56:56.854418 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:56.836147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.8.195.131:35191)
0812 16:56:56.836371 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:56.836376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:56.836403 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:56:56.836608 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:56.836612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:56.836683 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:56.836774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:56.838668 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.839412 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.841388 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:56.841845 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:56.841944 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:56.847840 (+ 5896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:56.847853 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:56.847870 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:56:56.847913 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:56.851354 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.852124 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.852131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.852133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.852276 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:56:56.852718 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:56:56.852722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:56:56.852725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:56:56.853110 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:56.853540 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:56.853829 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 16:56:56.854079 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:56:56.855185 11024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:56.982717 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10796
I20260812 16:56:57.009892 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 10030
I20260812 16:56:57.011144 10171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:57.123747 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 10030
2026-08-12T16:56:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5261 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T16:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:56:57Z Disabled control of system clock
I20260812 16:56:57.226426 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45875
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:56:57.571803 11148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:57.572117 11148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:57.572261 11148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:57.579651 11148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:56:57.579789 11148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:57.579885 11148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:57.579996 11148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:56:57.580080 11148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:56:57.589440 11148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:57.590771 11148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:57.592656 11148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:57.605090 11155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:57.605069 11160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:57.605342 11154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:57.605921 11148 server_base.cc:1034] running on GCE node
I20260812 16:56:57.606753 11148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:57.608212 11148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:57.609452 11148 hybrid_clock.cc:648] HybridClock initialized: now 1786553817609409 us; error 39 us; skew 500 ppm
I20260812 16:56:57.609815 11148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:56:57.612218 11148 webserver.cc:460] Webserver started at http://127.8.195.190:34595/ using document root <none> and password file <none>
I20260812 16:56:57.612931 11148 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:57.613070 11148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:57.613476 11148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:57.616472 11148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "11472afbf7fa4113b0e2dc48de3d45bf"
format_stamp: "Formatted at 2026-08-12 16:56:57 on dist-test-slave-z8x9"
server_key: "9bb5f07ec57bdec2154ff312bf25924c"
server_key_iv: "31325c49ea2d63d1f0aaf0b88c0594e0"
server_key_version: "encryptionkey@0"
I20260812 16:56:57.617381 11148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "11472afbf7fa4113b0e2dc48de3d45bf"
format_stamp: "Formatted at 2026-08-12 16:56:57 on dist-test-slave-z8x9"
server_key: "9bb5f07ec57bdec2154ff312bf25924c"
server_key_iv: "31325c49ea2d63d1f0aaf0b88c0594e0"
server_key_version: "encryptionkey@0"
I20260812 16:56:57.622550 11148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:56:57.626006 11172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:57.627259 11148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:56:57.627465 11148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "11472afbf7fa4113b0e2dc48de3d45bf"
format_stamp: "Formatted at 2026-08-12 16:56:57 on dist-test-slave-z8x9"
server_key: "9bb5f07ec57bdec2154ff312bf25924c"
server_key_iv: "31325c49ea2d63d1f0aaf0b88c0594e0"
server_key_version: "encryptionkey@0"
I20260812 16:56:57.627686 11148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:57.643959 11148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:57.644758 11148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:57.645056 11148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:57.663640 11148 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:45875
I20260812 16:56:57.663662 11275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:45875 every 8 connection(s)
I20260812 16:56:57.665045 11148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:56:57.666509 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11148
I20260812 16:56:57.666692 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:56:57.667273 8974 external_mini_cluster.cc:1428] Setting key b19fda54ef51f4e83f65d938950fb866
I20260812 16:56:57.669942 11276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:57.683666 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:57.669266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.8.195.190:45875)
0812 16:56:57.670357 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:57.670373 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:57.670426 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:56:57.672772 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:57.672801 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:57.673489 (+ 688us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:57.673976 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:57.674554 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:57.676652 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:57.678199 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:57.679263 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:57.679455 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:57.680667 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:57.680679 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:57.680699 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:57.681525 (+ 826us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:57.681626 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:57.681661 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:57.681799 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:57.681969 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 16:56:57.682336 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":344,"threads_started":1}
I20260812 16:56:57.684975 11276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: Bootstrap starting.
I20260812 16:56:57.688396 11276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:57.689478 11276 log.cc:824] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:57.692173 11276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: No bootstrap required, opened a new log
I20260812 16:56:57.699738 11276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } }
I20260812 16:56:57.700136 11276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:57.700274 11276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11472afbf7fa4113b0e2dc48de3d45bf, State: Initialized, Role: FOLLOWER
I20260812 16:56:57.701104 11276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } }
I20260812 16:56:57.701504 11276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:56:57.701691 11276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:56:57.701898 11276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:57.704202 11276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } }
I20260812 16:56:57.704854 11276 leader_election.cc:302] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11472afbf7fa4113b0e2dc48de3d45bf; no voters:
I20260812 16:56:57.705565 11276 leader_election.cc:288] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:56:57.705798 11285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:56:57.706735 11285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [term 1 LEADER]: Becoming Leader. State: Replica: 11472afbf7fa4113b0e2dc48de3d45bf, State: Running, Role: LEADER
I20260812 16:56:57.707315 11285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } }
I20260812 16:56:57.707923 11276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:56:57.711951 11287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11472afbf7fa4113b0e2dc48de3d45bf. Latest consensus state: current_term: 1 leader_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } } }
I20260812 16:56:57.710888 11286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11472afbf7fa4113b0e2dc48de3d45bf" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45875 } } }
I20260812 16:56:57.712467 11286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:57.713083 11287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf [sys.catalog]: This master's current role is: LEADER
I20260812 16:56:57.716161 11306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:56:57.721757 11306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:56:57.731334 11306 catalog_manager.cc:1347] Generated new cluster ID: f133c7317ab842c5a755b76ead4750cf
I20260812 16:56:57.731508 11306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:56:57.745644 11306 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:56:57.746999 11306 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:56:57.753981 11306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: Generated new TSK 0
I20260812 16:56:57.754818 11306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:56:57.768451 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--builtin_ntp_servers=127.8.195.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:56:58.136027 11326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:58.136320 11326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:58.136463 11326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:58.143757 11326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:58.143877 11326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:58.144055 11326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:56:58.153781 11326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:58.155150 11326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:58.156970 11326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:58.170169 11347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:58.170209 11351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:58.170457 11345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:58.171309 11326 server_base.cc:1034] running on GCE node
I20260812 16:56:58.171878 11326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:58.172766 11326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:58.173970 11326 hybrid_clock.cc:648] HybridClock initialized: now 1786553818173917 us; error 42 us; skew 500 ppm
I20260812 16:56:58.174240 11326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:56:58.176561 11326 webserver.cc:460] Webserver started at http://127.8.195.129:37167/ using document root <none> and password file <none>
I20260812 16:56:58.177281 11326 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:58.177397 11326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:58.177750 11326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:58.180590 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "e84a08b1e2ac46b18b9c6fd431ad075a"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "8737486c0fec1885a3323d93867b31a3"
server_key_iv: "9fcbc2923de398053ca1ec4dba290ca0"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.181425 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "e84a08b1e2ac46b18b9c6fd431ad075a"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "8737486c0fec1885a3323d93867b31a3"
server_key_iv: "9fcbc2923de398053ca1ec4dba290ca0"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.186729 11326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:56:58.190214 11363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.191709 11326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:56:58.191902 11326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "e84a08b1e2ac46b18b9c6fd431ad075a"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "8737486c0fec1885a3323d93867b31a3"
server_key_iv: "9fcbc2923de398053ca1ec4dba290ca0"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.192142 11326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:58.210902 11326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:58.212026 11326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:58.212463 11326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:58.214267 11326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:58.216133 11326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:58.216231 11326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.216372 11326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:58.216464 11326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.251699 11326 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:45275
I20260812 16:56:58.251786 11579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:45275 every 8 connection(s)
I20260812 16:56:58.253101 11326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:56:58.257578 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11326
I20260812 16:56:58.257776 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:56:58.258262 8974 external_mini_cluster.cc:1428] Setting key ad1d624625c632af891817b9ac511b89
I20260812 16:56:58.261377 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--builtin_ntp_servers=127.8.195.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:56:58.266736 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:58.255893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:47599 (local address 127.8.195.190:45875)
0812 16:56:58.256429 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:58.256434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:58.257659 (+ 1225us) server_negotiation.cc:407] Connection header received
0812 16:56:58.259922 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:58.259927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:58.259992 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:58.260134 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:58.261403 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.262198 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.263798 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.264203 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.264353 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:58.265847 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:58.265854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:58.265857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:58.265876 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:58.265917 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:58.265929 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:58.266029 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:58.266270 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 16:56:58.266376 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":315,"threads_started":1}
I20260812 16:56:58.267735 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:58.256510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45875 (local address 127.8.195.129:47599)
0812 16:56:58.257496 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:58.257533 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:58.259608 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:58.260452 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:58.260466 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:58.260576 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:58.261161 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:58.261237 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.262377 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.262383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:58.263644 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:58.263657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.264440 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.264448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:58.264692 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:58.265494 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:56:58.265497 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:58.265540 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:56:58.265625 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:58.266030 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:58.266045 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:58.266055 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:56:58.266241 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 16:56:58.266646 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":276,"threads_started":1}
I20260812 16:56:58.269496 11583 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45875
I20260812 16:56:58.269815 11583 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:58.270588 11583 heartbeater.cc:511] Master 127.8.195.190:45875 requested a full tablet report, sending...
I20260812 16:56:58.272917 11210 ts_manager.cc:194] Registered new tserver with Master: e84a08b1e2ac46b18b9c6fd431ad075a (127.8.195.129:45275)
I20260812 16:56:58.274667 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.129:47599
W20260812 16:56:58.607080 11590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:58.607429 11590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:58.607594 11590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:58.615079 11590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:58.615231 11590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:58.615413 11590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:56:58.624838 11590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:58.626197 11590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:58.628127 11590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:58.641554 11603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:58.641484 11606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:58.641530 11604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:58.642951 11590 server_base.cc:1034] running on GCE node
I20260812 16:56:58.643558 11590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:58.644495 11590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:58.645728 11590 hybrid_clock.cc:648] HybridClock initialized: now 1786553818645686 us; error 52 us; skew 500 ppm
I20260812 16:56:58.646062 11590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:56:58.648299 11590 webserver.cc:460] Webserver started at http://127.8.195.130:43963/ using document root <none> and password file <none>
I20260812 16:56:58.648994 11590 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:58.649128 11590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:58.649497 11590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:58.652266 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "854ab0f903b7480c97d683bac91b7868"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "d02bf006171b0b29d81ae34960fdfcdb"
server_key_iv: "ebf73fea7fd7f7552bdfe2165b5b979c"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.653066 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "854ab0f903b7480c97d683bac91b7868"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "d02bf006171b0b29d81ae34960fdfcdb"
server_key_iv: "ebf73fea7fd7f7552bdfe2165b5b979c"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.658294 11590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:56:58.662571 11619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.664724 11590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260812 16:56:58.664960 11590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "854ab0f903b7480c97d683bac91b7868"
format_stamp: "Formatted at 2026-08-12 16:56:58 on dist-test-slave-z8x9"
server_key: "d02bf006171b0b29d81ae34960fdfcdb"
server_key_iv: "ebf73fea7fd7f7552bdfe2165b5b979c"
server_key_version: "encryptionkey@0"
I20260812 16:56:58.665310 11590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:58.687112 11590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:58.687929 11590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:58.688207 11590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:58.689155 11590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:58.690833 11590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:58.690936 11590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.691025 11590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:58.691080 11590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:58.725082 11590 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:46415
I20260812 16:56:58.725133 11853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:46415 every 8 connection(s)
I20260812 16:56:58.726758 11590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:56:58.735848 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11590
I20260812 16:56:58.736058 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:56:58.736569 8974 external_mini_cluster.cc:1428] Setting key fa01da2c3d312103f230c9634ad7d6f1
I20260812 16:56:58.739678 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:58.729503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:35859 (local address 127.8.195.190:45875)
0812 16:56:58.729720 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:58.729724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:58.730968 (+ 1244us) server_negotiation.cc:407] Connection header received
0812 16:56:58.732917 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:58.732922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:58.732993 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:58.733082 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:58.734173 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.734952 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.736591 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.737237 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.737356 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:58.738850 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:58.738858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:58.738863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:58.738887 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:58.738928 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:58.738943 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:58.739043 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:58.739261 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 16:56:58.739375 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 16:56:58.739985 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--builtin_ntp_servers=127.8.195.148:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:56:58.740682 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:58.729935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45875 (local address 127.8.195.130:35859)
0812 16:56:58.730813 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:58.730856 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:58.732649 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:58.733366 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:58.733375 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:58.733458 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:58.734008 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:58.734024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.735089 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.735092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:58.736440 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:58.736452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:58.737489 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:58.737500 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:58.737713 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:58.738459 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:56:58.738462 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:58.738525 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:56:58.738627 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:58.739049 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:58.739058 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:58.739063 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:56:58.739210 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 16:56:58.739498 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":269,"threads_started":1}
I20260812 16:56:58.742187 11855 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45875
I20260812 16:56:58.742512 11855 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:58.743414 11855 heartbeater.cc:511] Master 127.8.195.190:45875 requested a full tablet report, sending...
I20260812 16:56:58.744969 11210 ts_manager.cc:194] Registered new tserver with Master: 854ab0f903b7480c97d683bac91b7868 (127.8.195.130:46415)
I20260812 16:56:58.745851 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.130:35859
W20260812 16:56:59.091739 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:56:59.092037 11858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:56:59.092123 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:56:59.099611 11858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:56:59.099726 11858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:56:59.099833 11858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:56:59.109151 11858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:56:59.110430 11858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:56:59.112174 11858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:56:59.125701 11870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:59.125784 11875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:56:59.125759 11871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:56:59.126866 11858 server_base.cc:1034] running on GCE node
I20260812 16:56:59.127574 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:56:59.128566 11858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:56:59.129805 11858 hybrid_clock.cc:648] HybridClock initialized: now 1786553819129757 us; error 62 us; skew 500 ppm
I20260812 16:56:59.130131 11858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:56:59.132640 11858 webserver.cc:460] Webserver started at http://127.8.195.131:40811/ using document root <none> and password file <none>
I20260812 16:56:59.133455 11858 fs_manager.cc:359] Metadata directory not provided
I20260812 16:56:59.133606 11858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:56:59.133915 11858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:56:59.136716 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "10f141ef02974888aea176f09e5d6321"
format_stamp: "Formatted at 2026-08-12 16:56:59 on dist-test-slave-z8x9"
server_key: "df3e54330da1c2dda93e87f7af536d6f"
server_key_iv: "e80f1dd3e875f0b9582b1aa127ef4c3f"
server_key_version: "encryptionkey@0"
I20260812 16:56:59.137548 11858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "10f141ef02974888aea176f09e5d6321"
format_stamp: "Formatted at 2026-08-12 16:56:59 on dist-test-slave-z8x9"
server_key: "df3e54330da1c2dda93e87f7af536d6f"
server_key_iv: "e80f1dd3e875f0b9582b1aa127ef4c3f"
server_key_version: "encryptionkey@0"
I20260812 16:56:59.142601 11858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 16:56:59.146052 11890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:59.147423 11858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:56:59.147568 11858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "10f141ef02974888aea176f09e5d6321"
format_stamp: "Formatted at 2026-08-12 16:56:59 on dist-test-slave-z8x9"
server_key: "df3e54330da1c2dda93e87f7af536d6f"
server_key_iv: "e80f1dd3e875f0b9582b1aa127ef4c3f"
server_key_version: "encryptionkey@0"
I20260812 16:56:59.147753 11858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:56:59.173038 11858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:56:59.173923 11858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:56:59.174202 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:56:59.175153 11858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:56:59.176607 11858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:56:59.176699 11858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:59.176781 11858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:56:59.176839 11858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:56:59.210979 11858 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:40027
I20260812 16:56:59.211063 12125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:40027 every 8 connection(s)
I20260812 16:56:59.212317 11858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:56:59.214901 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 11858
I20260812 16:56:59.215096 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:56:59.215682 8974 external_mini_cluster.cc:1428] Setting key f5147e19278be8f78314addd85794745
I20260812 16:56:59.225871 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.215049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:32773 (local address 127.8.195.190:45875)
0812 16:56:59.215281 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.215289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.217457 (+ 2168us) server_negotiation.cc:407] Connection header received
0812 16:56:59.219868 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.219873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.219941 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.220029 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:59.221058 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.221859 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.223360 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.223746 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.223840 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.225118 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:59.225123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:59.225127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:59.225145 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:59.225233 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:59.225251 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:59.225329 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.225539 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.225618 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:56:59.226861 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.215675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45875 (local address 127.8.195.131:32773)
0812 16:56:59.217260 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.217311 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.219617 (+ 2306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.220230 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.220238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.220330 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:56:59.220911 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.220928 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.221999 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.222003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.223228 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.223240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.223935 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.223939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.224137 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.224814 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:56:59.224817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:56:59.224857 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:56:59.224939 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:56:59.225332 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:56:59.225341 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:56:59.225346 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.225508 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.225797 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":439,"threads_started":1}
I20260812 16:56:59.228279 12127 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45875
I20260812 16:56:59.228641 12127 heartbeater.cc:464] Registering TS with master...
I20260812 16:56:59.229476 12127 heartbeater.cc:511] Master 127.8.195.190:45875 requested a full tablet report, sending...
I20260812 16:56:59.231124 11210 ts_manager.cc:194] Registered new tserver with Master: 10f141ef02974888aea176f09e5d6321 (127.8.195.131:40027)
I20260812 16:56:59.231762 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:56:59.232205 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.131:32773
I20260812 16:56:59.241842 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.236315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.8.195.190:45875)
0812 16:56:59.236515 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.236555 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.236921 (+ 366us) server_negotiation.cc:407] Connection header received
0812 16:56:59.237047 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.237051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.237118 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.237268 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:59.237738 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.238402 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.239710 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.240089 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.240178 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.241161 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:59.241208 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:59.241217 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:59.241237 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:59.241279 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:59.241292 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:59.241360 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.241541 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.241611 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:56:59.245476 11210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:56:59.252442 11210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:56:59.275266 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.264466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:46415 (local address 127.0.0.1:36462)
0812 16:56:59.265336 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.265378 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.265551 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.267306 (+ 1755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.267313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.267345 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:59.267786 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.267801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.269710 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.269715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.272481 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.272496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.274257 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.274266 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.274495 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.274549 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.274701 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.274901 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":33,"thread_start_us":864,"threads_started":1}
I20260812 16:56:59.275321 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.264669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36462 (local address 127.8.195.130:46415)
0812 16:56:59.266589 (+ 1920us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.266678 (+ 89us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.266708 (+ 30us) server_negotiation.cc:407] Connection header received
0812 16:56:59.266815 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.266829 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.267028 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.267275 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:59.268016 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.269577 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.272759 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.273843 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.273924 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.274089 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.274765 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.274971 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1766,"thread_start_us":1623,"threads_started":1}
I20260812 16:56:59.275560 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.264993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.8.195.131:40027)
0812 16:56:59.266161 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.266170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.267269 (+ 1099us) server_negotiation.cc:407] Connection header received
0812 16:56:59.267389 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.267398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.267622 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.267822 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:59.268728 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.270339 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.272648 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.274112 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.274238 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.274467 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.275040 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.275199 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":1592,"threads_started":1}
I20260812 16:56:59.276314 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.264615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40027 (local address 127.0.0.1:40562)
0812 16:56:59.266991 (+ 2376us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.267009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.267130 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.268153 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.268157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.268181 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:59.268541 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.268554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.270475 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.270483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.272391 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.272406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.274607 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.274612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.274826 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.274858 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.275833 (+ 975us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.275968 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2223,"mutex_wait_us":1,"thread_start_us":2074,"threads_started":1}
I20260812 16:56:59.278196 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.267095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45275 (local address 127.0.0.1:49790)
0812 16:56:59.267899 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.267918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.268016 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.269441 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.269446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.269474 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:59.269958 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.269971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.271929 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.271936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.274112 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.274128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.276279 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.276286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.277632 (+ 1346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.277656 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.277774 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.277898 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":633,"threads_started":1}
I20260812 16:56:59.278695 11583 heartbeater.cc:503] Master 127.8.195.190:45875 was elected leader, sending a full tablet report...
I20260812 16:56:59.279009 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.267523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.8.195.129:45275)
0812 16:56:59.268564 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.268570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.268591 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:56:59.268706 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.268713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.268932 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.269127 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:59.270160 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.271763 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.274337 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.276079 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.276231 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.277049 (+ 818us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.277869 (+ 820us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.278032 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":773,"threads_started":1}
I20260812 16:56:59.279889 11974 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (DEFAULT_TABLE table=test-table [id=adc966bece2a47d780e2faeab27039c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:59.281751 11974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4cf21ea3d24adca625e4e46eaa9b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:59.279701 11708 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (DEFAULT_TABLE table=test-table [id=adc966bece2a47d780e2faeab27039c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:59.282720 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4cf21ea3d24adca625e4e46eaa9b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:59.282454 11466 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (DEFAULT_TABLE table=test-table [id=adc966bece2a47d780e2faeab27039c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:56:59.283799 11466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4cf21ea3d24adca625e4e46eaa9b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:56:59.293799 12172 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Bootstrap starting.
I20260812 16:56:59.298143 12172 tablet_bootstrap.cc:654] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:59.298486 12176 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Bootstrap starting.
I20260812 16:56:59.299738 12172 log.cc:824] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:59.300266 12175 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Bootstrap starting.
I20260812 16:56:59.303052 12176 tablet_bootstrap.cc:654] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:59.303373 12175 tablet_bootstrap.cc:654] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Neither blocks nor log segments found. Creating new log.
I20260812 16:56:59.304629 12175 log.cc:824] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:59.304672 12176 log.cc:824] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:56:59.307762 12175 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: No bootstrap required, opened a new log
I20260812 16:56:59.308054 12175 ts_tablet_manager.cc:1397] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 16:56:59.308094 12172 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: No bootstrap required, opened a new log
I20260812 16:56:59.308154 12176 tablet_bootstrap.cc:492] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: No bootstrap required, opened a new log
I20260812 16:56:59.308372 12176 ts_tablet_manager.cc:1397] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260812 16:56:59.308373 12172 ts_tablet_manager.cc:1397] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260812 16:56:59.315687 12175 raft_consensus.cc:348] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.316211 12175 raft_consensus.cc:374] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:59.316326 12175 raft_consensus.cc:729] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f141ef02974888aea176f09e5d6321, State: Initialized, Role: FOLLOWER
I20260812 16:56:59.317011 12175 consensus_queue.cc:260] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.318352 12127 heartbeater.cc:503] Master 127.8.195.190:45875 was elected leader, sending a full tablet report...
I20260812 16:56:59.319267 12172 raft_consensus.cc:348] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.319272 12176 raft_consensus.cc:348] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.319705 12172 raft_consensus.cc:374] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:59.319705 12176 raft_consensus.cc:374] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:56:59.319818 12172 raft_consensus.cc:729] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 854ab0f903b7480c97d683bac91b7868, State: Initialized, Role: FOLLOWER
I20260812 16:56:59.319813 12176 raft_consensus.cc:729] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84a08b1e2ac46b18b9c6fd431ad075a, State: Initialized, Role: FOLLOWER
I20260812 16:56:59.319931 12175 ts_tablet_manager.cc:1428] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260812 16:56:59.320439 12172 consensus_queue.cc:260] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.320444 12176 consensus_queue.cc:260] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.322364 12172 ts_tablet_manager.cc:1428] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260812 16:56:59.322711 12176 ts_tablet_manager.cc:1428] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260812 16:56:59.323423 11855 heartbeater.cc:503] Master 127.8.195.190:45875 was elected leader, sending a full tablet report...
I20260812 16:56:59.400913 12187 raft_consensus.cc:482] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:56:59.401332 12187 raft_consensus.cc:504] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.403170 12187 leader_election.cc:288] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f141ef02974888aea176f09e5d6321 (127.8.195.131:40027), e84a08b1e2ac46b18b9c6fd431ad075a (127.8.195.129:45275)
I20260812 16:56:59.410858 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.403645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:40629 (local address 127.8.195.131:40027)
0812 16:56:59.403900 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.403910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.404449 (+ 539us) server_negotiation.cc:407] Connection header received
0812 16:56:59.404677 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.404683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.404754 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.404868 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:59.405665 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.406673 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.408763 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.409856 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.409982 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.410031 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.410444 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.410581 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 16:56:59.410862 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.403616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40027 (local address 127.8.195.130:40629)
0812 16:56:59.404299 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.404340 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.404533 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.405051 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.405055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.405080 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:59.405507 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.405522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.406829 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.406835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.408571 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.408585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.410054 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.410064 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.410259 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.410298 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.410398 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.410532 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":387,"threads_started":1}
I20260812 16:56:59.411193 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.404433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:45275 (local address 127.8.195.130:39463)
0812 16:56:59.405331 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0812 16:56:59.405346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 16:56:59.405460 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:56:59.405799 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:56:59.405804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:56:59.405829 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:56:59.406183 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.406195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.407387 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.407391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.408914 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:56:59.408929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.410380 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.410389 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:56:59.410569 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.410599 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 16:56:59.410692 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 16:56:59.410816 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":517,"threads_started":1}
I20260812 16:56:59.411346 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.404723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:39463 (local address 127.8.195.129:45275)
0812 16:56:59.404947 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.404953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.405436 (+ 483us) server_negotiation.cc:407] Connection header received
0812 16:56:59.405577 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.405584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.405683 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.405749 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:56:59.406345 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.407240 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.409109 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.410174 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.410345 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.410425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.410739 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.410881 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 16:56:59.412299 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4cf21ea3d24adca625e4e46eaa9b68" candidate_uuid: "854ab0f903b7480c97d683bac91b7868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f141ef02974888aea176f09e5d6321" is_pre_election: true
I20260812 16:56:59.412569 11500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4cf21ea3d24adca625e4e46eaa9b68" candidate_uuid: "854ab0f903b7480c97d683bac91b7868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" is_pre_election: true
I20260812 16:56:59.412822 12025 raft_consensus.cc:2454] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 854ab0f903b7480c97d683bac91b7868 in term 0.
I20260812 16:56:59.413093 11500 raft_consensus.cc:2454] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 854ab0f903b7480c97d683bac91b7868 in term 0.
I20260812 16:56:59.413645 11623 leader_election.cc:302] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f141ef02974888aea176f09e5d6321, 854ab0f903b7480c97d683bac91b7868; no voters:
I20260812 16:56:59.414062 12187 raft_consensus.cc:2781] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:56:59.414224 12187 raft_consensus.cc:482] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:56:59.414345 12187 raft_consensus.cc:3037] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:59.416572 12187 raft_consensus.cc:504] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.417356 12187 leader_election.cc:288] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [CANDIDATE]: Term 1 election: Requested vote from peers 10f141ef02974888aea176f09e5d6321 (127.8.195.131:40027), e84a08b1e2ac46b18b9c6fd431ad075a (127.8.195.129:45275)
I20260812 16:56:59.417737 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4cf21ea3d24adca625e4e46eaa9b68" candidate_uuid: "854ab0f903b7480c97d683bac91b7868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f141ef02974888aea176f09e5d6321"
I20260812 16:56:59.417785 11500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4cf21ea3d24adca625e4e46eaa9b68" candidate_uuid: "854ab0f903b7480c97d683bac91b7868" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a"
I20260812 16:56:59.417968 12025 raft_consensus.cc:3037] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:59.418001 11500 raft_consensus.cc:3037] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:56:59.421286 11500 raft_consensus.cc:2454] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 854ab0f903b7480c97d683bac91b7868 in term 1.
I20260812 16:56:59.421286 12025 raft_consensus.cc:2454] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 854ab0f903b7480c97d683bac91b7868 in term 1.
I20260812 16:56:59.421710 11623 leader_election.cc:302] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f141ef02974888aea176f09e5d6321, 854ab0f903b7480c97d683bac91b7868; no voters:
I20260812 16:56:59.422032 12187 raft_consensus.cc:2781] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:56:59.422438 12187 raft_consensus.cc:686] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 1 LEADER]: Becoming Leader. State: Replica: 854ab0f903b7480c97d683bac91b7868, State: Running, Role: LEADER
I20260812 16:56:59.422910 12187 consensus_queue.cc:237] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } }
I20260812 16:56:59.427707 11209 catalog_manager.cc:5495] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 reported cstate change: term changed from 0 to 1, leader changed from <none> to 854ab0f903b7480c97d683bac91b7868 (127.8.195.130). New cstate: current_term: 1 leader_uuid: "854ab0f903b7480c97d683bac91b7868" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "854ab0f903b7480c97d683bac91b7868" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 46415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 } health_report { overall_health: UNKNOWN } } }
W20260812 16:56:59.464112 12128 tablet.cc:2369] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:59.468066 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.463040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.8.195.130:46415)
0812 16:56:59.463219 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.463223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.463239 (+ 16us) server_negotiation.cc:407] Connection header received
0812 16:56:59.463345 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.463348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.463436 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.463569 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:56:59.463962 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.464747 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.466197 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.466602 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.466694 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.467124 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:56:59.467604 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:56:59.467711 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.467807 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.467867 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:56:59.476296 12025 raft_consensus.cc:1261] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 1 FOLLOWER]: Refusing update from remote peer 854ab0f903b7480c97d683bac91b7868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:59.476282 11500 raft_consensus.cc:1261] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 1 FOLLOWER]: Refusing update from remote peer 854ab0f903b7480c97d683bac91b7868: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:56:59.477366 12191 consensus_queue.cc:1035] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f141ef02974888aea176f09e5d6321" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:56:59.477738 12187 consensus_queue.cc:1035] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e84a08b1e2ac46b18b9c6fd431ad075a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 45275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 16:56:59.478895 11856 tablet.cc:2369] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:56:59.488715 12200 mvcc.cc:205] Tried to move back new op lower bound from 7317724444564307968 to 7317724444359909376. Current Snapshot: MvccSnapshot[applied={T|T < 7317724444564307968}]
I20260812 16:56:59.500394 11210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:59.500665 11210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:56:59.504304 11210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11472afbf7fa4113b0e2dc48de3d45bf: Sending DeleteTablet for 3 replicas of tablet 4a4cf21ea3d24adca625e4e46eaa9b68
I20260812 16:56:59.505519 11708 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:59 UTC) from {username='slave'} at 127.0.0.1:36462
I20260812 16:56:59.505748 11974 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:59 UTC) from {username='slave'} at 127.0.0.1:40562
I20260812 16:56:59.506304 11466 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4cf21ea3d24adca625e4e46eaa9b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:56:59 UTC) from {username='slave'} at 127.0.0.1:49790
I20260812 16:56:59.506481 12215 tablet_replica.cc:330] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: stopping tablet replica
I20260812 16:56:59.506925 12215 raft_consensus.cc:2229] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:56:59.507607 12215 raft_consensus.cc:2258] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:59.508638 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11326
I20260812 16:56:59.516132 12219 tablet_replica.cc:330] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: stopping tablet replica
I20260812 16:56:59.516597 12219 raft_consensus.cc:2229] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:59.517311 12219 raft_consensus.cc:2258] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:59.519773 12219 ts_tablet_manager.cc:1905] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:59.521185 12215 ts_tablet_manager.cc:1905] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:59.530078 12219 ts_tablet_manager.cc:1918] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:59.530308 12219 log.cc:1196] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/4a4cf21ea3d24adca625e4e46eaa9b68
I20260812 16:56:59.531021 12219 ts_tablet_manager.cc:1939] T 4a4cf21ea3d24adca625e4e46eaa9b68 P e84a08b1e2ac46b18b9c6fd431ad075a: Deleting consensus metadata
I20260812 16:56:59.533833 11174 catalog_manager.cc:4841] TS e84a08b1e2ac46b18b9c6fd431ad075a (127.8.195.129:45275): tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (table test-table [id=adc966bece2a47d780e2faeab27039c8]) successfully deleted
I20260812 16:56:59.534912 12216 tablet_replica.cc:330] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: stopping tablet replica
I20260812 16:56:59.535526 12216 raft_consensus.cc:2229] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:56:59.536188 12216 raft_consensus.cc:2258] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:56:59.538786 12215 ts_tablet_manager.cc:1918] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:59.538987 12215 log.cc:1196] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/4a4cf21ea3d24adca625e4e46eaa9b68
I20260812 16:56:59.539510 12215 ts_tablet_manager.cc:1939] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 854ab0f903b7480c97d683bac91b7868: Deleting consensus metadata
I20260812 16:56:59.541391 11173 catalog_manager.cc:4841] TS 854ab0f903b7480c97d683bac91b7868 (127.8.195.130:46415): tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (table test-table [id=adc966bece2a47d780e2faeab27039c8]) successfully deleted
I20260812 16:56:59.542698 12216 ts_tablet_manager.cc:1905] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:56:59.545492 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.509406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.8.195.129:45275)
0812 16:56:59.509633 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.509642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.537235 (+ 27593us) server_negotiation.cc:407] Connection header received
0812 16:56:59.537494 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.537499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.537607 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.537700 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:59.538324 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.539480 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.541427 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.542046 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.542151 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.543749 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:59.543762 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:59.543779 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:59.544570 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:59.544638 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:59.544665 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:59.544788 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.544966 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.545084 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 16:56:59.546415 11572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:59.549813 12216 ts_tablet_manager.cc:1918] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:56:59.550022 12216 log.cc:1196] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/4a4cf21ea3d24adca625e4e46eaa9b68
I20260812 16:56:59.551119 12216 ts_tablet_manager.cc:1939] T 4a4cf21ea3d24adca625e4e46eaa9b68 P 10f141ef02974888aea176f09e5d6321: Deleting consensus metadata
I20260812 16:56:59.553628 11174 catalog_manager.cc:4841] TS 10f141ef02974888aea176f09e5d6321 (127.8.195.131:40027): tablet 4a4cf21ea3d24adca625e4e46eaa9b68 (table test-table [id=adc966bece2a47d780e2faeab27039c8]) successfully deleted
I20260812 16:56:59.665072 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11326
I20260812 16:56:59.698180 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11590
I20260812 16:56:59.707376 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.699006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.8.195.130:46415)
0812 16:56:59.699221 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.699228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.699540 (+ 312us) server_negotiation.cc:407] Connection header received
0812 16:56:59.699736 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.699741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.699859 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.699963 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:59.700474 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.701771 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.703791 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.704260 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.704371 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.705828 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:59.705838 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:59.705855 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:59.706511 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:59.706564 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:59.706588 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:59.706687 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.706878 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.706969 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:56:59.708155 11845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:59.822821 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11590
I20260812 16:56:59.853003 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11858
I20260812 16:56:59.859759 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:56:59.853832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.8.195.131:40027)
0812 16:56:59.854049 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 16:56:59.854053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:56:59.854072 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:56:59.854211 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:56:59.854215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:56:59.854297 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:56:59.854396 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:56:59.854768 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.855478 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.856713 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:56:59.857094 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:56:59.857407 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:56:59.858419 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:56:59.858427 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:56:59.858443 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:56:59.859015 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:56:59.859066 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:56:59.859086 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:56:59.859176 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:56:59.859293 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 16:56:59.859390 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 16:56:59.860632 12114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:56:59.951426 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11858
I20260812 16:56:59.976150 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 11148
I20260812 16:56:59.977375 11270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:00.087599 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 11148
2026-08-12T16:57:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2967 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:00 dist-test-slave-z8x9 krb5kdc[12230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:00 dist-test-slave-z8x9 krb5kdc[12230](info): set up 2 sockets
Aug 12 16:57:00 dist-test-slave-z8x9 krb5kdc[12230](info): commencing operation
krb5kdc: starting...
W20260812 16:57:01.544363 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:01 dist-test-slave-z8x9 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:01.880035 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:46511
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:02.231106 12247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:02.231406 12247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:02.231498 12247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:02.238927 12247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:02.239051 12247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:02.239099 12247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:02.239158 12247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:02.239208 12247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:02.249466 12247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:46511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:02.250880 12247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:02.252143 12247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:02.264636 12253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:02.264586 12255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:02.264580 12264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:02.265508 12247 server_base.cc:1034] running on GCE node
I20260812 16:57:02.266352 12247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:02.267802 12247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:02.269025 12247 hybrid_clock.cc:648] HybridClock initialized: now 1786553822268958 us; error 48 us; skew 500 ppm
I20260812 16:57:02.269459 12247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:02 dist-test-slave-z8x9 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553822, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:02.283182 12247 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:02.284884 12247 webserver.cc:460] Webserver started at http://127.8.195.190:39425/ using document root <none> and password file <none>
I20260812 16:57:02.285681 12247 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:02.285805 12247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:02.286082 12247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:02.289155 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "8e70890f5d874a95830f9bc56e7d9396"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "eb4702c991cdf9f7e0ffa4e52b10708b"
server_key_iv: "67de6316cfbf044a7707bb1cf385fd18"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.289975 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "8e70890f5d874a95830f9bc56e7d9396"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "eb4702c991cdf9f7e0ffa4e52b10708b"
server_key_iv: "67de6316cfbf044a7707bb1cf385fd18"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.295475 12247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:02.299190 12278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:02.300580 12247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:57:02.300734 12247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "8e70890f5d874a95830f9bc56e7d9396"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "eb4702c991cdf9f7e0ffa4e52b10708b"
server_key_iv: "67de6316cfbf044a7707bb1cf385fd18"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.300966 12247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:02.326680 12247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:02.330862 12247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:02.331106 12247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:02.351123 12247 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:46511
I20260812 16:57:02.351155 12394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:46511 every 8 connection(s)
I20260812 16:57:02.352569 12247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:02.356477 12396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:02.362399 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 12247
I20260812 16:57:02.362618 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:02.363251 8974 external_mini_cluster.cc:1428] Setting key c16d28e3bbe7d3ddcad58ecf013a5aa1
I20260812 16:57:02.368630 12396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: Bootstrap starting.
I20260812 16:57:02.371852 12396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:02.373320 12396 log.cc:824] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:02.376621 12396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: No bootstrap required, opened a new log
Aug 12 16:57:02 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:02.385057 12396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } }
I20260812 16:57:02.385594 12396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:02.385732 12396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e70890f5d874a95830f9bc56e7d9396, State: Initialized, Role: FOLLOWER
I20260812 16:57:02.386510 12396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } }
I20260812 16:57:02.386801 12396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:02.386963 12396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:02.387236 12396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:02.387892 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:02.365362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.8.195.190:46511)
0812 16:57:02.366365 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:02.366380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:02.366424 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:57:02.368227 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:02.368256 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:02.368667 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:02.369115 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:02.370742 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:02.372119 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:02.373648 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:02.374814 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:02.375008 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:02.379564 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:02.379572 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:02.379588 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:02.379632 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:02.383194 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:02.384008 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:02.384016 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:02.384022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:02.384211 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:02.384634 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:02.384638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:02.384640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:02.385256 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:02.385587 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:02.386155 (+ 568us) server_negotiation.cc:299] Negotiation successful
0812 16:57:02.386573 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":461,"threads_started":1}
I20260812 16:57:02.390376 12396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } }
I20260812 16:57:02.390995 12396 leader_election.cc:302] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e70890f5d874a95830f9bc56e7d9396; no voters:
I20260812 16:57:02.391638 12396 leader_election.cc:288] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:02.391793 12405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:02.392549 12405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [term 1 LEADER]: Becoming Leader. State: Replica: 8e70890f5d874a95830f9bc56e7d9396, State: Running, Role: LEADER
I20260812 16:57:02.393106 12405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } }
I20260812 16:57:02.393874 12396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:02.396071 12408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e70890f5d874a95830f9bc56e7d9396. Latest consensus state: current_term: 1 leader_uuid: "8e70890f5d874a95830f9bc56e7d9396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } } }
I20260812 16:57:02.396544 12408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:02.396095 12407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e70890f5d874a95830f9bc56e7d9396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e70890f5d874a95830f9bc56e7d9396" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 46511 } } }
I20260812 16:57:02.398340 12407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:02.402035 12417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:02.409035 12417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:02.418977 12417 catalog_manager.cc:1347] Generated new cluster ID: d7226d1100c640229ed39616e6c79bb9
I20260812 16:57:02.419162 12417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:02.441653 12417 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:02.443198 12417 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:02.451561 12417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: Generated new TSK 0
I20260812 16:57:02.452468 12417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:57:02.532959 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:02.891165 12453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:02.891516 12453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:02.891695 12453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:02.899322 12453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:02.899485 12453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:02.899662 12453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:57:02.910008 12453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:02.911379 12453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:02.912863 12453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:02.926530 12469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:02.926561 12468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:02.926610 12472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:02.927556 12453 server_base.cc:1034] running on GCE node
I20260812 16:57:02.928162 12453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:02.929095 12453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:02.930341 12453 hybrid_clock.cc:648] HybridClock initialized: now 1786553822930267 us; error 56 us; skew 500 ppm
I20260812 16:57:02.930641 12453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:02 dist-test-slave-z8x9 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553822, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:02.934069 12453 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:57:02.935572 12453 webserver.cc:460] Webserver started at http://127.8.195.129:34979/ using document root <none> and password file <none>
I20260812 16:57:02.936362 12453 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:02.936501 12453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:02.936838 12453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:02.939827 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "337442e4686c496ca871aa28addb518a"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "412a1ab126d358586983b57235442311"
server_key_iv: "aa5daed1bd4e4883810baa3e83e07f72"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.940610 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "337442e4686c496ca871aa28addb518a"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "412a1ab126d358586983b57235442311"
server_key_iv: "aa5daed1bd4e4883810baa3e83e07f72"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.946096 12453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 16:57:02.949745 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:02.951411 12453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:57:02.951604 12453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "337442e4686c496ca871aa28addb518a"
format_stamp: "Formatted at 2026-08-12 16:57:02 on dist-test-slave-z8x9"
server_key: "412a1ab126d358586983b57235442311"
server_key_iv: "aa5daed1bd4e4883810baa3e83e07f72"
server_key_version: "encryptionkey@0"
I20260812 16:57:02.951828 12453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:02.972285 12453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:02.976575 12453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:02.976950 12453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:02.978048 12453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:02.979444 12453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:02.979549 12453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:02.979722 12453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:02.979802 12453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:03.014983 12453 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:41403
I20260812 16:57:03.015074 12738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:41403 every 8 connection(s)
I20260812 16:57:03.016307 12453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:57:03.018040 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 12453
I20260812 16:57:03.018244 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:57:03.018683 8974 external_mini_cluster.cc:1428] Setting key 6b00309b0cf9727243a99f581f6e093b
Aug 12 16:57:03 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553822, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:03.038801 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:03.018992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:37497 (local address 127.8.195.190:46511)
0812 16:57:03.019475 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:03.019482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:03.020605 (+ 1123us) server_negotiation.cc:407] Connection header received
0812 16:57:03.022619 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:03.022628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:03.022742 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:03.022871 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:03.024481 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.025634 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.027184 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.027917 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.028055 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:03.031814 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:03.031823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:03.031830 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:03.031877 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:03.035295 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:03.036296 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:03.036301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:03.036303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:03.036433 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:03.036947 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:03.036951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:03.036952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:03.037314 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:03.037431 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:03.038333 (+ 902us) server_negotiation.cc:299] Negotiation successful
0812 16:57:03.038506 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":230,"threads_started":1}
I20260812 16:57:03.040398 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:03.019614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:46511 (local address 127.8.195.129:37497)
0812 16:57:03.020430 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:03.020467 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:03.022334 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:03.023074 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:03.023081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:03.023644 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:03.024302 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:03.024320 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.025799 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.025803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:03.027032 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:03.027044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.028133 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.028137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:03.028359 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:03.029078 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:03.029105 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:03.031579 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:03.035448 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:03.035460 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:03.035479 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:03.036169 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:03.036549 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:03.036556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:03.036562 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:03.036838 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:03.037441 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:03.037452 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:03.037926 (+ 474us) client_negotiation.cc:769] Sending connection context
0812 16:57:03.038254 (+ 328us) client_negotiation.cc:240] Negotiation successful
0812 16:57:03.038881 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":330,"threads_started":1}
I20260812 16:57:03.042089 12739 heartbeater.cc:347] Connected to a master server at 127.8.195.190:46511
I20260812 16:57:03.042420 12739 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:03.043237 12739 heartbeater.cc:511] Master 127.8.195.190:46511 requested a full tablet report, sending...
I20260812 16:57:03.045563 12315 ts_manager.cc:194] Registered new tserver with Master: 337442e4686c496ca871aa28addb518a (127.8.195.129:41403)
I20260812 16:57:03.047096 12315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:37497
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:57:03.093374 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:03.448119 12748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:03.448416 12748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:03.448580 12748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:03.456068 12748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:03.456220 12748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:03.456415 12748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:57:03.466665 12748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:03.468025 12748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:03.469507 12748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:03.482810 12758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:03.482755 12761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:03.483289 12757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:03.483695 12748 server_base.cc:1034] running on GCE node
I20260812 16:57:03.484284 12748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:03.485219 12748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:03.486435 12748 hybrid_clock.cc:648] HybridClock initialized: now 1786553823486397 us; error 35 us; skew 500 ppm
I20260812 16:57:03.486763 12748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:03 dist-test-slave-z8x9 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:03.489948 12748 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:57:03.491575 12748 webserver.cc:460] Webserver started at http://127.8.195.130:44361/ using document root <none> and password file <none>
I20260812 16:57:03.492292 12748 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:03.492455 12748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:03.492796 12748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:03.495996 12748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "ca1a65357464403d84ebb0af4e7fff85"
format_stamp: "Formatted at 2026-08-12 16:57:03 on dist-test-slave-z8x9"
server_key: "4eb2b6355f25b94431061b0bbf543cd3"
server_key_iv: "2f28c8265908d5336c90e5bec5d96df0"
server_key_version: "encryptionkey@0"
I20260812 16:57:03.496819 12748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "ca1a65357464403d84ebb0af4e7fff85"
format_stamp: "Formatted at 2026-08-12 16:57:03 on dist-test-slave-z8x9"
server_key: "4eb2b6355f25b94431061b0bbf543cd3"
server_key_iv: "2f28c8265908d5336c90e5bec5d96df0"
server_key_version: "encryptionkey@0"
I20260812 16:57:03.501976 12748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 16:57:03.505676 12778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:03.507095 12748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:57:03.507243 12748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "ca1a65357464403d84ebb0af4e7fff85"
format_stamp: "Formatted at 2026-08-12 16:57:03 on dist-test-slave-z8x9"
server_key: "4eb2b6355f25b94431061b0bbf543cd3"
server_key_iv: "2f28c8265908d5336c90e5bec5d96df0"
server_key_version: "encryptionkey@0"
I20260812 16:57:03.507421 12748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:03.545648 12748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:03.550096 12748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:03.550374 12748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:03.551349 12748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:03.552712 12748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:03.552801 12748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:03.552891 12748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:03.552934 12748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:03.588450 12748 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:40407
I20260812 16:57:03.588511 13023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:40407 every 8 connection(s)
I20260812 16:57:03.589864 12748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:57:03.598364 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 12748
I20260812 16:57:03.598553 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:57:03.599089 8974 external_mini_cluster.cc:1428] Setting key 64989c1f750f936e1b2c3121957e16f9
Aug 12 16:57:03 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553823, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:03.613198 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:03.592463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:55781 (local address 127.8.195.190:46511)
0812 16:57:03.592982 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:03.592986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:03.593851 (+ 865us) server_negotiation.cc:407] Connection header received
0812 16:57:03.595852 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:03.595858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:03.595958 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:03.596080 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:03.597675 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.598719 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.600531 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.601126 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.601288 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:03.605425 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:03.605434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:03.605441 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:03.605511 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:03.609346 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:03.610173 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:03.610182 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:03.610188 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:03.610367 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:03.610906 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:03.610914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:03.610917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:03.611310 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:03.611464 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:03.612399 (+ 935us) server_negotiation.cc:299] Negotiation successful
0812 16:57:03.612676 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":291,"threads_started":1}
I20260812 16:57:03.613971 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:03.592896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:46511 (local address 127.8.195.130:55781)
0812 16:57:03.593685 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:03.593732 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:03.595565 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:03.596315 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:03.596323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:03.596849 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:03.597497 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:03.597514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.598885 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.598889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:03.600361 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:03.600376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:03.601389 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:03.601395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:03.601599 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:03.602358 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:03.602380 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:03.605121 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:03.609515 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:03.609522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:03.609536 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:03.610004 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:03.610523 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:03.610527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:03.610530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:03.610744 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:03.611482 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:03.611494 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:03.611894 (+ 400us) client_negotiation.cc:769] Sending connection context
0812 16:57:03.612228 (+ 334us) client_negotiation.cc:240] Negotiation successful
0812 16:57:03.612664 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":351,"threads_started":1}
I20260812 16:57:03.615549 13024 heartbeater.cc:347] Connected to a master server at 127.8.195.190:46511
I20260812 16:57:03.615878 13024 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:03.616684 13024 heartbeater.cc:511] Master 127.8.195.190:46511 requested a full tablet report, sending...
I20260812 16:57:03.618506 12315 ts_manager.cc:194] Registered new tserver with Master: ca1a65357464403d84ebb0af4e7fff85 (127.8.195.130:40407)
I20260812 16:57:03.619865 12315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:55781
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:57:03.672834 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:04.031445 13036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:04.031741 13036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:04.031838 13036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:04.039201 13036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:04.039332 13036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:04.039449 13036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:57:04.049674 13036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:46511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:04.050355 12739 heartbeater.cc:503] Master 127.8.195.190:46511 was elected leader, sending a full tablet report...
I20260812 16:57:04.051215 13036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:04.052909 13036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:04.066692 13045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:04.066618 13049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:04.066744 13051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:04.067891 13036 server_base.cc:1034] running on GCE node
I20260812 16:57:04.068542 13036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:04.069602 13036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:04.070840 13036 hybrid_clock.cc:648] HybridClock initialized: now 1786553824070809 us; error 52 us; skew 500 ppm
I20260812 16:57:04.071166 13036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:04 dist-test-slave-z8x9 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:04.074549 13036 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:57:04.076100 13036 webserver.cc:460] Webserver started at http://127.8.195.131:34745/ using document root <none> and password file <none>
I20260812 16:57:04.076862 13036 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:04.077018 13036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:04.077435 13036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:04.080513 13036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "e19f4d8dbc40481bbbce21cc87048e19"
format_stamp: "Formatted at 2026-08-12 16:57:04 on dist-test-slave-z8x9"
server_key: "5ebcc634db3051a3e8e02d9967b01844"
server_key_iv: "b76c339b996be1c77db281fad539a631"
server_key_version: "encryptionkey@0"
I20260812 16:57:04.081413 13036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "e19f4d8dbc40481bbbce21cc87048e19"
format_stamp: "Formatted at 2026-08-12 16:57:04 on dist-test-slave-z8x9"
server_key: "5ebcc634db3051a3e8e02d9967b01844"
server_key_iv: "b76c339b996be1c77db281fad539a631"
server_key_version: "encryptionkey@0"
I20260812 16:57:04.086884 13036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:57:04.090582 13060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:04.092113 13036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:04.092367 13036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "e19f4d8dbc40481bbbce21cc87048e19"
format_stamp: "Formatted at 2026-08-12 16:57:04 on dist-test-slave-z8x9"
server_key: "5ebcc634db3051a3e8e02d9967b01844"
server_key_iv: "b76c339b996be1c77db281fad539a631"
server_key_version: "encryptionkey@0"
I20260812 16:57:04.092597 13036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:04.112166 13036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:04.117393 13036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:04.117739 13036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:04.118788 13036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:04.120268 13036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:04.120347 13036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:04.120486 13036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:04.120538 13036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:04.153837 13036 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:34211
I20260812 16:57:04.153884 13308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:34211 every 8 connection(s)
I20260812 16:57:04.155191 13036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:57:04.158782 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13036
I20260812 16:57:04.159040 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:57:04.159651 8974 external_mini_cluster.cc:1428] Setting key 7496ec1ef11a7b89c2ca07b34d9a326e
Aug 12 16:57:04 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553824, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:04.176555 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.157952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:34933 (local address 127.8.195.190:46511)
0812 16:57:04.158410 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.158418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.159623 (+ 1205us) server_negotiation.cc:407] Connection header received
0812 16:57:04.161826 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.161834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.161927 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.162018 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:04.163785 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.164558 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.166144 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.166560 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.166657 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.170087 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:04.170091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:04.170095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:04.170128 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:04.172951 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.173890 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.173894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.173897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.174048 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.174546 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.174550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.174552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.174963 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:04.175137 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.175925 (+ 788us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.176149 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":209,"threads_started":1}
I20260812 16:57:04.177559 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.158536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:46511 (local address 127.8.195.131:34933)
0812 16:57:04.159417 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.159474 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.161569 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.162310 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.162325 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.163076 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:04.163628 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.163645 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.164695 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.164699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.166000 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.166012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.166751 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.166757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.166960 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.167614 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:04.167641 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:04.169891 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:04.173094 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:04.173101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:04.173114 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:04.173756 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:04.174178 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:04.174183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:04.174186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:04.174407 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:04.175102 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:04.175111 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:04.175540 (+ 429us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.175824 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.176365 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":351,"threads_started":1}
I20260812 16:57:04.178879 13310 heartbeater.cc:347] Connected to a master server at 127.8.195.190:46511
I20260812 16:57:04.179175 13310 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:04.179891 13310 heartbeater.cc:511] Master 127.8.195.190:46511 requested a full tablet report, sending...
I20260812 16:57:04.181319 12315 ts_manager.cc:194] Registered new tserver with Master: e19f4d8dbc40481bbbce21cc87048e19 (127.8.195.131:34211)
I20260812 16:57:04.182143 12315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:34933
I20260812 16:57:04.187753 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:57:04.206159 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.192345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.8.195.190:46511)
0812 16:57:04.192518 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.192523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.192938 (+ 415us) server_negotiation.cc:407] Connection header received
0812 16:57:04.193071 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.193075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.193136 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.193278 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:04.194780 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.195668 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.197046 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.197614 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.197720 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.200474 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:04.200479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:04.200482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:04.200515 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:04.203246 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.204099 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.204109 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.204111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.204239 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.204707 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.204711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.204713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.205049 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:04.205336 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.205760 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.205914 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 16:57:04.210350 12315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:57:04.217459 12315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:57:04.254220 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.230631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:34211 (local address 127.0.0.1:51022)
0812 16:57:04.242310 (+ 11679us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.242333 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.242468 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.243399 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.243411 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.243458 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:04.244054 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.244072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.247001 (+ 2929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.247007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.250951 (+ 3944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.250965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.252908 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.252916 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.253231 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.253355 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.253502 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.253710 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11536,"mutex_wait_us":645,"thread_start_us":10745,"threads_started":1}
I20260812 16:57:04.255039 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.230934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.8.195.131:34211)
0812 16:57:04.241814 (+ 10880us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.241828 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.242617 (+ 789us) server_negotiation.cc:407] Connection header received
0812 16:57:04.242759 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.242767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.243068 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.243328 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:04.244389 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.246806 (+ 2417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.251139 (+ 4333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.252657 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.252805 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.253067 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.254489 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.254670 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10653,"thread_start_us":10494,"threads_started":1}
I20260812 16:57:04.255812 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.230624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40407 (local address 127.0.0.1:44768)
0812 16:57:04.233079 (+ 2455us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.233117 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.233437 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.247878 (+ 14441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.247883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.247909 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:04.248402 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.248425 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.250689 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.250698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.252917 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.252937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.254912 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.254921 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.255101 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.255147 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.255237 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.255371 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2222,"mutex_wait_us":1,"thread_start_us":2031,"threads_started":1}
I20260812 16:57:04.256556 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.230934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44768 (local address 127.8.195.130:40407)
0812 16:57:04.246935 (+ 16001us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.246944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.246977 (+ 33us) server_negotiation.cc:407] Connection header received
0812 16:57:04.247097 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.247105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.247400 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.248654 (+ 1254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:04.248912 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.250535 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.253172 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.254671 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.254824 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.255093 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.255885 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.256130 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15788,"thread_start_us":16640,"threads_started":1}
I20260812 16:57:04.258038 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.242830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.8.195.129:41403)
0812 16:57:04.249689 (+ 6859us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.249710 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.250158 (+ 448us) server_negotiation.cc:407] Connection header received
0812 16:57:04.250370 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.250377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.250571 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.250820 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:04.251512 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.252539 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.255176 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.256652 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.256794 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.257011 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.257519 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.257712 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6676,"thread_start_us":6503,"threads_started":1}
I20260812 16:57:04.259271 13163 tablet_service.cc:1467] Processing CreateTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 (DEFAULT_TABLE table=test-table [id=d6ba8ac76655488cb0ee9aaed2545005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:04.259635 12880 tablet_service.cc:1467] Processing CreateTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 (DEFAULT_TABLE table=test-table [id=d6ba8ac76655488cb0ee9aaed2545005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:04.260890 13163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71e51bc01e249d0aa98a0dbbe8a7d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:04.260895 12880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71e51bc01e249d0aa98a0dbbe8a7d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:04.261461 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.243830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54342)
0812 16:57:04.250020 (+ 6190us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.250037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.250187 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.250835 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.250840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.250876 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:04.251314 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.251325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.252690 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.252697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.254957 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.254972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.257122 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.257141 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.257336 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.257358 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.260895 (+ 3537us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.261059 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6068,"thread_start_us":5898,"threads_started":1}
I20260812 16:57:04.265337 12587 tablet_service.cc:1467] Processing CreateTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 (DEFAULT_TABLE table=test-table [id=d6ba8ac76655488cb0ee9aaed2545005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:04.266950 12587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b71e51bc01e249d0aa98a0dbbe8a7d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:04.272317 13358 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Bootstrap starting.
I20260812 16:57:04.275691 13358 tablet_bootstrap.cc:654] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:04.275795 13361 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Bootstrap starting.
I20260812 16:57:04.277158 13358 log.cc:824] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:04.279501 13361 tablet_bootstrap.cc:654] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:04.280592 13358 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: No bootstrap required, opened a new log
I20260812 16:57:04.280776 13361 log.cc:824] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:04.280890 13358 ts_tablet_manager.cc:1397] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 16:57:04.283180 13367 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Bootstrap starting.
I20260812 16:57:04.286597 13367 tablet_bootstrap.cc:654] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:04.287711 13367 log.cc:824] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:04.288125 13361 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: No bootstrap required, opened a new log
I20260812 16:57:04.288475 13361 ts_tablet_manager.cc:1397] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260812 16:57:04.290772 13367 tablet_bootstrap.cc:492] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: No bootstrap required, opened a new log
I20260812 16:57:04.291043 13367 ts_tablet_manager.cc:1397] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 16:57:04.293041 13358 raft_consensus.cc:348] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.293529 13358 raft_consensus.cc:374] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:04.293675 13358 raft_consensus.cc:729] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19f4d8dbc40481bbbce21cc87048e19, State: Initialized, Role: FOLLOWER
I20260812 16:57:04.294353 13358 consensus_queue.cc:260] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.298139 13310 heartbeater.cc:503] Master 127.8.195.190:46511 was elected leader, sending a full tablet report...
I20260812 16:57:04.298979 13358 ts_tablet_manager.cc:1428] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Time spent starting tablet: real 0.018s user 0.009s sys 0.007s
I20260812 16:57:04.300108 13361 raft_consensus.cc:348] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.300350 13367 raft_consensus.cc:348] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.300616 13361 raft_consensus.cc:374] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:04.300724 13361 raft_consensus.cc:729] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1a65357464403d84ebb0af4e7fff85, State: Initialized, Role: FOLLOWER
I20260812 16:57:04.300750 13367 raft_consensus.cc:374] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:04.300876 13367 raft_consensus.cc:729] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337442e4686c496ca871aa28addb518a, State: Initialized, Role: FOLLOWER
I20260812 16:57:04.301512 13367 consensus_queue.cc:260] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.301542 13361 consensus_queue.cc:260] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.303401 13367 ts_tablet_manager.cc:1428] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260812 16:57:04.304944 13024 heartbeater.cc:503] Master 127.8.195.190:46511 was elected leader, sending a full tablet report...
I20260812 16:57:04.306540 13361 ts_tablet_manager.cc:1428] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260812 16:57:04.310839 13373 raft_consensus.cc:482] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:57:04.311151 13373 raft_consensus.cc:504] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.313033 13373 leader_election.cc:288] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e19f4d8dbc40481bbbce21cc87048e19 (127.8.195.131:34211), ca1a65357464403d84ebb0af4e7fff85 (127.8.195.130:40407)
I20260812 16:57:04.321928 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.313598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:38971 (local address 127.8.195.131:34211)
0812 16:57:04.313829 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.313838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.314315 (+ 477us) server_negotiation.cc:407] Connection header received
0812 16:57:04.314551 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.314563 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.314623 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.314741 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:04.315384 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.316528 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.319431 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.320732 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.320906 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.320969 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.321512 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.321648 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:57:04.321928 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.313505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:49573 (local address 127.8.195.130:40407)
0812 16:57:04.313733 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.313741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.314726 (+ 985us) server_negotiation.cc:407] Connection header received
0812 16:57:04.314932 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.314936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.314992 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.315070 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:04.315686 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.316740 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.319427 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.320732 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.320897 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.320969 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.321512 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.321648 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:57:04.322194 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.313919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:40407 (local address 127.8.195.129:49573)
0812 16:57:04.314617 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.314631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.314779 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.315126 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.315132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.315166 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:04.315522 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.315530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.316874 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.316878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.319245 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.319261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.321057 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.321063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.321334 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.321379 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.321467 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.321669 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":173,"thread_start_us":468,"threads_started":1}
I20260812 16:57:04.322217 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.313509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:34211 (local address 127.8.195.129:38971)
0812 16:57:04.314192 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.314209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:04.314399 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:04.314780 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:04.314785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:04.314816 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:04.315206 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.315224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.316655 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.316659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.319245 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:04.319261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.321041 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.321049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:04.321253 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.321366 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 16:57:04.321467 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 16:57:04.321669 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":104,"thread_start_us":619,"threads_started":1}
I20260812 16:57:04.323194 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b71e51bc01e249d0aa98a0dbbe8a7d07" candidate_uuid: "337442e4686c496ca871aa28addb518a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19f4d8dbc40481bbbce21cc87048e19" is_pre_election: true
I20260812 16:57:04.323267 12923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b71e51bc01e249d0aa98a0dbbe8a7d07" candidate_uuid: "337442e4686c496ca871aa28addb518a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1a65357464403d84ebb0af4e7fff85" is_pre_election: true
I20260812 16:57:04.323753 13199 raft_consensus.cc:2454] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337442e4686c496ca871aa28addb518a in term 0.
I20260812 16:57:04.323774 12923 raft_consensus.cc:2454] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337442e4686c496ca871aa28addb518a in term 0.
I20260812 16:57:04.324445 12497 leader_election.cc:302] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337442e4686c496ca871aa28addb518a, ca1a65357464403d84ebb0af4e7fff85; no voters:
I20260812 16:57:04.324831 13373 raft_consensus.cc:2781] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:57:04.324988 13373 raft_consensus.cc:482] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:57:04.325116 13373 raft_consensus.cc:3037] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:04.327548 13373 raft_consensus.cc:504] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.328214 13373 leader_election.cc:288] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [CANDIDATE]: Term 1 election: Requested vote from peers e19f4d8dbc40481bbbce21cc87048e19 (127.8.195.131:34211), ca1a65357464403d84ebb0af4e7fff85 (127.8.195.130:40407)
I20260812 16:57:04.328675 13199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b71e51bc01e249d0aa98a0dbbe8a7d07" candidate_uuid: "337442e4686c496ca871aa28addb518a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19f4d8dbc40481bbbce21cc87048e19"
I20260812 16:57:04.328742 12923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b71e51bc01e249d0aa98a0dbbe8a7d07" candidate_uuid: "337442e4686c496ca871aa28addb518a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1a65357464403d84ebb0af4e7fff85"
I20260812 16:57:04.328910 13199 raft_consensus.cc:3037] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:04.328986 12923 raft_consensus.cc:3037] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:04.331509 13199 raft_consensus.cc:2454] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337442e4686c496ca871aa28addb518a in term 1.
I20260812 16:57:04.331539 12923 raft_consensus.cc:2454] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337442e4686c496ca871aa28addb518a in term 1.
I20260812 16:57:04.332013 12485 leader_election.cc:302] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337442e4686c496ca871aa28addb518a, e19f4d8dbc40481bbbce21cc87048e19; no voters:
I20260812 16:57:04.332391 13373 raft_consensus.cc:2781] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:04.332864 13373 raft_consensus.cc:686] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 1 LEADER]: Becoming Leader. State: Replica: 337442e4686c496ca871aa28addb518a, State: Running, Role: LEADER
I20260812 16:57:04.333415 13373 consensus_queue.cc:237] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } }
I20260812 16:57:04.338214 12315 catalog_manager.cc:5495] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a reported cstate change: term changed from 0 to 1, leader changed from <none> to 337442e4686c496ca871aa28addb518a (127.8.195.129). New cstate: current_term: 1 leader_uuid: "337442e4686c496ca871aa28addb518a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "337442e4686c496ca871aa28addb518a" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 41403 } health_report { overall_health: HEALTHY } } }
W20260812 16:57:04.341954 13025 tablet.cc:2369] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:57:04.369994 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.363897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.8.195.129:41403)
0812 16:57:04.364101 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.364107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.364198 (+ 91us) server_negotiation.cc:407] Connection header received
0812 16:57:04.364383 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.364389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.364514 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.364673 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:57:04.365044 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.366143 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.367567 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.367981 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.368077 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.368708 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:57:04.369319 (+ 611us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:57:04.369460 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.369633 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.369672 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:57:04.378609 12923 raft_consensus.cc:1261] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 1 FOLLOWER]: Refusing update from remote peer 337442e4686c496ca871aa28addb518a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:04.378619 13199 raft_consensus.cc:1261] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 1 FOLLOWER]: Refusing update from remote peer 337442e4686c496ca871aa28addb518a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:04.379705 13373 consensus_queue.cc:1035] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19f4d8dbc40481bbbce21cc87048e19" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 34211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:04.380018 13383 consensus_queue.cc:1035] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1a65357464403d84ebb0af4e7fff85" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 40407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:04.394767 13386 mvcc.cc:205] Tried to move back new op lower bound from 7317724464643403776 to 7317724464473255936. Current Snapshot: MvccSnapshot[applied={T|T < 7317724464643403776 or (T in {7317724464643403776})}]
I20260812 16:57:04.396968 13388 mvcc.cc:205] Tried to move back new op lower bound from 7317724464643403776 to 7317724464473255936. Current Snapshot: MvccSnapshot[applied={T|T < 7317724464643403776}]
I20260812 16:57:04.406879 12315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:04.407193 12315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:04.412297 12315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e70890f5d874a95830f9bc56e7d9396: Sending DeleteTablet for 3 replicas of tablet b71e51bc01e249d0aa98a0dbbe8a7d07
I20260812 16:57:04.413725 13163 tablet_service.cc:1514] Processing DeleteTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:04 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:51022
I20260812 16:57:04.414232 12587 tablet_service.cc:1514] Processing DeleteTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:04 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:54342
I20260812 16:57:04.414520 12880 tablet_service.cc:1514] Processing DeleteTablet for tablet b71e51bc01e249d0aa98a0dbbe8a7d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:04 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:44768
I20260812 16:57:04.414708 13400 tablet_replica.cc:330] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: stopping tablet replica
I20260812 16:57:04.415158 13400 raft_consensus.cc:2229] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:04.415699 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12453
I20260812 16:57:04.415910 13400 raft_consensus.cc:2258] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:04.422349 13400 ts_tablet_manager.cc:1905] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:04.428949 13404 tablet_replica.cc:330] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: stopping tablet replica
I20260812 16:57:04.429634 13404 raft_consensus.cc:2229] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:04.430290 13404 raft_consensus.cc:2258] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:04.431218 13403 tablet_replica.cc:330] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: stopping tablet replica
Aug 12 16:57:04 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:57:04.431783 13403 raft_consensus.cc:2229] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:57:04.433379 13404 ts_tablet_manager.cc:1905] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:04.433495 13400 ts_tablet_manager.cc:1918] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:04.433897 13400 log.cc:1196] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/b71e51bc01e249d0aa98a0dbbe8a7d07
I20260812 16:57:04.434134 13403 raft_consensus.cc:2258] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:04.434880 13400 ts_tablet_manager.cc:1939] T b71e51bc01e249d0aa98a0dbbe8a7d07 P e19f4d8dbc40481bbbce21cc87048e19: Deleting consensus metadata
I20260812 16:57:04.436985 12280 catalog_manager.cc:4841] TS e19f4d8dbc40481bbbce21cc87048e19 (127.8.195.131:34211): tablet b71e51bc01e249d0aa98a0dbbe8a7d07 (table test-table [id=d6ba8ac76655488cb0ee9aaed2545005]) successfully deleted
I20260812 16:57:04.439669 13403 ts_tablet_manager.cc:1905] T b71e51bc01e249d0aa98a0dbbe8a7d07 P 337442e4686c496ca871aa28addb518a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:04.440485 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.417965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54350 (local address 127.8.195.129:41403)
0812 16:57:04.418206 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.418216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.419769 (+ 1553us) server_negotiation.cc:407] Connection header received
0812 16:57:04.420119 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.420128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.420241 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.420331 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:04.422473 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.423623 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.425593 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.426324 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.426428 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.433113 (+ 6685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:04.433132 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:04.433152 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:04.433234 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:04.437569 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.438397 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.438403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.438406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.438536 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.438960 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.438964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.438966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.439328 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:04.439677 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.439990 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.440201 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:57:04.441848 12732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:04.444509 13404 ts_tablet_manager.cc:1918] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:04.445022 13404 log.cc:1196] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/b71e51bc01e249d0aa98a0dbbe8a7d07
I20260812 16:57:04.445669 13404 ts_tablet_manager.cc:1939] T b71e51bc01e249d0aa98a0dbbe8a7d07 P ca1a65357464403d84ebb0af4e7fff85: Deleting consensus metadata
I20260812 16:57:04.447866 12285 catalog_manager.cc:4841] TS ca1a65357464403d84ebb0af4e7fff85 (127.8.195.130:40407): tablet b71e51bc01e249d0aa98a0dbbe8a7d07 (table test-table [id=d6ba8ac76655488cb0ee9aaed2545005]) successfully deleted
I20260812 16:57:04.542136 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 12453
I20260812 16:57:04.567229 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12748
W20260812 16:57:04.567176 12280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:41403 (error 108)
I20260812 16:57:04.569458 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.568889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54354)
0812 16:57:04.569079 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.569249 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 16:57:04.569768 12280 catalog_manager.cc:4584] TS 337442e4686c496ca871aa28addb518a (127.8.195.129:41403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b71e51bc01e249d0aa98a0dbbe8a7d07: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Aug 12 16:57:04 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:57:04.583346 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.568125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.8.195.130:40407)
0812 16:57:04.568311 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.568317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.568515 (+ 198us) server_negotiation.cc:407] Connection header received
0812 16:57:04.568741 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.568750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.568849 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.568965 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:04.570602 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.571346 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.572584 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.572954 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.573044 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.577563 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:04.577576 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:04.577592 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:04.577626 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:04.580589 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.581406 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.581414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.581416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.581545 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.581903 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.581906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.581908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.582301 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:04.582523 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.582770 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.582938 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 16:57:04.584039 13014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:04.623870 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.623307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54356)
0812 16:57:04.623559 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.623679 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260812 16:57:04.625133 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.624749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54372)
0812 16:57:04.624909 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.624998 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 16:57:04.677326 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 12748
I20260812 16:57:04.685731 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.685242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54374)
0812 16:57:04.685437 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.685534 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 16:57:04.686937 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.686571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54380)
0812 16:57:04.686712 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.686794 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 16:57:04.707129 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13036
Aug 12 16:57:04 dist-test-slave-z8x9 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:57:04.723775 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.708049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.8.195.131:34211)
0812 16:57:04.708267 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:04.708275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:04.708372 (+ 97us) server_negotiation.cc:407] Connection header received
0812 16:57:04.708599 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:04.708609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:04.708675 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:04.708831 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:04.710319 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.711078 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.712445 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:04.712855 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:04.712997 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:04.717868 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:04.717885 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:04.717902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:04.717937 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:04.720655 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.721531 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.721539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.721542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.721719 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:04.722118 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:04.722123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:04.722127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:04.722508 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:04.722830 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:04.723185 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 16:57:04.723466 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:57:04.724490 13299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:04.767743 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.767250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54386)
0812 16:57:04.767487 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.767583 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 16:57:04.768882 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.768450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54388)
0812 16:57:04.768678 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.768752 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 16:57:04.819772 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13036
I20260812 16:57:04.847613 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 12247
I20260812 16:57:04.848791 12383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:04.960640 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:04.960167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:41403 (local address 127.0.0.1:54396)
0812 16:57:04.960378 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:04.960470 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 16:57:04.961262 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 12247
2026-08-12T16:57:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4872 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T16:57:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:05Z Disabled control of system clock
I20260812 16:57:05.068085 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42633
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:57:05.423794 13439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:05.424083 13439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:05.424252 13439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:05.431764 13439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:05.431882 13439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:05.431969 13439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:05.432076 13439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:05.432166 13439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:05.441787 13439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:05.443171 13439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:05.445034 13439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:05.457713 13455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:05.457598 13449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:05.457643 13448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:05.458779 13439 server_base.cc:1034] running on GCE node
I20260812 16:57:05.459637 13439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:05.461355 13439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:05.462633 13439 hybrid_clock.cc:648] HybridClock initialized: now 1786553825462566 us; error 57 us; skew 500 ppm
I20260812 16:57:05.463037 13439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:05.465771 13439 webserver.cc:460] Webserver started at http://127.8.195.190:42503/ using document root <none> and password file <none>
I20260812 16:57:05.466511 13439 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:05.466657 13439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:05.467021 13439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:05.470297 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "0c52db08a30b4f36ab00d73ab8d75b55"
format_stamp: "Formatted at 2026-08-12 16:57:05 on dist-test-slave-z8x9"
server_key: "28ede32030b3d90c6722a74528bc6c70"
server_key_iv: "6a7df856f7c94d68707123bf31eaf701"
server_key_version: "encryptionkey@0"
I20260812 16:57:05.471159 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "0c52db08a30b4f36ab00d73ab8d75b55"
format_stamp: "Formatted at 2026-08-12 16:57:05 on dist-test-slave-z8x9"
server_key: "28ede32030b3d90c6722a74528bc6c70"
server_key_iv: "6a7df856f7c94d68707123bf31eaf701"
server_key_version: "encryptionkey@0"
I20260812 16:57:05.476696 13439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:57:05.480482 13469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:05.481910 13439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:57:05.482117 13439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "0c52db08a30b4f36ab00d73ab8d75b55"
format_stamp: "Formatted at 2026-08-12 16:57:05 on dist-test-slave-z8x9"
server_key: "28ede32030b3d90c6722a74528bc6c70"
server_key_iv: "6a7df856f7c94d68707123bf31eaf701"
server_key_version: "encryptionkey@0"
I20260812 16:57:05.482378 13439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:05.500347 13439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:05.501317 13439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:05.501621 13439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:05.521899 13439 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:42633
I20260812 16:57:05.521943 13577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:42633 every 8 connection(s)
I20260812 16:57:05.523352 13439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:05.527347 13584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:05.530133 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13439
I20260812 16:57:05.530326 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:05.530781 8974 external_mini_cluster.cc:1428] Setting key 02c7c90a1a99f3264d088d6f0296465a
I20260812 16:57:05.541476 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: Bootstrap starting.
I20260812 16:57:05.545862 13584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:05.547261 13584 log.cc:824] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:05.547750 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:05.532894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.8.195.190:42633)
0812 16:57:05.534050 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:05.534066 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:05.534119 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:57:05.536553 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:05.536589 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:05.537322 (+ 733us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:05.537814 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:05.538399 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:05.540455 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:05.542030 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:05.543081 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:05.543326 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:05.544475 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:05.544487 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:05.544505 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:05.545283 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:05.545414 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:05.545450 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:05.545574 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:05.545997 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 16:57:05.546380 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":403,"threads_started":1}
I20260812 16:57:05.551801 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: No bootstrap required, opened a new log
I20260812 16:57:05.559223 13584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } }
I20260812 16:57:05.559603 13584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:05.559675 13584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c52db08a30b4f36ab00d73ab8d75b55, State: Initialized, Role: FOLLOWER
I20260812 16:57:05.560254 13584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } }
I20260812 16:57:05.560472 13584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:05.560573 13584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:05.560745 13584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:05.563107 13584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } }
I20260812 16:57:05.563629 13584 leader_election.cc:302] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c52db08a30b4f36ab00d73ab8d75b55; no voters:
I20260812 16:57:05.564234 13584 leader_election.cc:288] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:05.564445 13606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:05.565231 13606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [term 1 LEADER]: Becoming Leader. State: Replica: 0c52db08a30b4f36ab00d73ab8d75b55, State: Running, Role: LEADER
I20260812 16:57:05.565730 13606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } }
I20260812 16:57:05.566462 13584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:05.568672 13608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } } }
I20260812 16:57:05.569147 13608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:05.570449 13610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c52db08a30b4f36ab00d73ab8d75b55. Latest consensus state: current_term: 1 leader_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c52db08a30b4f36ab00d73ab8d75b55" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42633 } } }
I20260812 16:57:05.570855 13610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:05.573645 13635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:05.578331 13635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:05.587365 13635 catalog_manager.cc:1347] Generated new cluster ID: 6876c0ec44404d27b66e6c6bda36c4f5
I20260812 16:57:05.587505 13635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:05.603111 13635 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:05.604666 13635 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:05.613253 13635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: Generated new TSK 0
I20260812 16:57:05.614128 13635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:05.622773 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--builtin_ntp_servers=127.8.195.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:57:05.994659 13658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:05.995020 13658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:05.995198 13658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:06.002745 13658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:06.002894 13658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:06.003078 13658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:57:06.012625 13658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:06.014056 13658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:06.015861 13658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:06.029683 13664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.029721 13665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.029793 13670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:06.030478 13658 server_base.cc:1034] running on GCE node
I20260812 16:57:06.031230 13658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:06.032160 13658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:06.033354 13658 hybrid_clock.cc:648] HybridClock initialized: now 1786553826033294 us; error 49 us; skew 500 ppm
I20260812 16:57:06.033700 13658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:06.036046 13658 webserver.cc:460] Webserver started at http://127.8.195.129:41133/ using document root <none> and password file <none>
I20260812 16:57:06.036765 13658 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:06.036886 13658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:06.037320 13658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:06.040253 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "5aa8d8d82f6f4387823f8017496d405e"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "f45a23d27d28a36d3bba05836829b4cd"
server_key_iv: "35f09a91d3806dd59028dbd2991bc49f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.041067 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "5aa8d8d82f6f4387823f8017496d405e"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "f45a23d27d28a36d3bba05836829b4cd"
server_key_iv: "35f09a91d3806dd59028dbd2991bc49f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.046132 13658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 16:57:06.049728 13678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.051127 13658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:57:06.051321 13658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "5aa8d8d82f6f4387823f8017496d405e"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "f45a23d27d28a36d3bba05836829b4cd"
server_key_iv: "35f09a91d3806dd59028dbd2991bc49f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.051568 13658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:06.074038 13658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:06.074952 13658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:06.075294 13658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:06.076375 13658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:06.078047 13658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:06.078141 13658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.078253 13658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:06.078336 13658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.111614 13658 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:36679
I20260812 16:57:06.111721 13927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:36679 every 8 connection(s)
I20260812 16:57:06.113026 13658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:57:06.119961 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13658
I20260812 16:57:06.120185 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:57:06.120641 8974 external_mini_cluster.cc:1428] Setting key de7009f85702894711902fa942039ee7
I20260812 16:57:06.123566 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--builtin_ntp_servers=127.8.195.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:57:06.126763 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:06.115937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34131 (local address 127.8.195.190:42633)
0812 16:57:06.116459 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:06.116466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:06.117731 (+ 1265us) server_negotiation.cc:407] Connection header received
0812 16:57:06.119562 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:06.119568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:06.119668 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:06.119804 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:06.121030 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.121973 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.123857 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.124260 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.124402 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:06.125890 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:06.125898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:06.125907 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:06.125932 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:06.125974 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:06.125987 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:06.126085 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:06.126313 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 16:57:06.126410 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":235,"threads_started":1}
I20260812 16:57:06.127816 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:06.116553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42633 (local address 127.8.195.129:34131)
0812 16:57:06.117566 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:06.117604 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:06.119259 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:06.120004 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:06.120012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:06.120101 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:06.120841 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:06.120861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.122165 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.122169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:06.123623 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:06.123667 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.124496 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.124503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:06.124751 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:06.125547 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:06.125550 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:06.125591 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:06.125687 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:06.126087 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:06.126102 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:06.126110 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 16:57:06.126304 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 16:57:06.126725 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":275,"threads_started":1}
I20260812 16:57:06.129519 13933 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42633
I20260812 16:57:06.129845 13933 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:06.130616 13933 heartbeater.cc:511] Master 127.8.195.190:42633 requested a full tablet report, sending...
I20260812 16:57:06.133013 13511 ts_manager.cc:194] Registered new tserver with Master: 5aa8d8d82f6f4387823f8017496d405e (127.8.195.129:36679)
I20260812 16:57:06.135231 13511 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.129:34131
W20260812 16:57:06.473541 13948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:06.473831 13948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:06.473922 13948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:06.481235 13948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:06.481348 13948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:06.481465 13948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:57:06.490851 13948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:06.492138 13948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:06.493978 13948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:06.508370 13957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.508332 13965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.508409 13959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:06.509142 13948 server_base.cc:1034] running on GCE node
I20260812 16:57:06.509824 13948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:06.510838 13948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:06.512050 13948 hybrid_clock.cc:648] HybridClock initialized: now 1786553826512008 us; error 36 us; skew 500 ppm
I20260812 16:57:06.512408 13948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:06.514919 13948 webserver.cc:460] Webserver started at http://127.8.195.130:41713/ using document root <none> and password file <none>
I20260812 16:57:06.515683 13948 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:06.515784 13948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:06.516175 13948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:06.519448 13948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "a36061dd81b54b2cb286e3739bf701ff"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "069b6ea33d30f3b718f9e6333769ee47"
server_key_iv: "310b841f073488e6cfc24502cb7faf3f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.520323 13948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "a36061dd81b54b2cb286e3739bf701ff"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "069b6ea33d30f3b718f9e6333769ee47"
server_key_iv: "310b841f073488e6cfc24502cb7faf3f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.527168 13948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260812 16:57:06.531699 13985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.533478 13948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 16:57:06.533643 13948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "a36061dd81b54b2cb286e3739bf701ff"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "069b6ea33d30f3b718f9e6333769ee47"
server_key_iv: "310b841f073488e6cfc24502cb7faf3f"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.533876 13948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:06.548832 13948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:06.549783 13948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:06.550099 13948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:06.551081 13948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:06.552553 13948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:06.552672 13948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.552810 13948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:06.552904 13948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:06.587463 13948 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:41895
I20260812 16:57:06.587579 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:41895 every 8 connection(s)
I20260812 16:57:06.588937 13948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:57:06.598296 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 13948
I20260812 16:57:06.598449 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:57:06.598938 8974 external_mini_cluster.cc:1428] Setting key 2cb14489171ad99d32d3cc191d43c46d
I20260812 16:57:06.601783 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--builtin_ntp_servers=127.8.195.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:57:06.602603 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:06.591679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:34461 (local address 127.8.195.190:42633)
0812 16:57:06.591895 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:06.591904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:06.593367 (+ 1463us) server_negotiation.cc:407] Connection header received
0812 16:57:06.595121 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:06.595125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:06.595193 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:06.595306 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:06.596337 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.597077 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.599126 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.599742 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.599910 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:06.601507 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:06.601514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:06.601527 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:06.601546 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:06.601589 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:06.601603 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:06.601722 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:06.602093 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 16:57:06.602205 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:57:06.603482 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:06.592229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42633 (local address 127.8.195.130:34461)
0812 16:57:06.593124 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:06.593222 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:06.594876 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:06.595495 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:06.595504 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:06.595598 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:06.596158 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:06.596175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.597350 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.597354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:06.598935 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:06.598950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:06.600016 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:06.600025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:06.600292 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:06.601096 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:06.601099 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:06.601148 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:06.601295 (+ 147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:06.601713 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:06.601724 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:06.601736 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 16:57:06.601956 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 16:57:06.602305 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260812 16:57:06.605095 14208 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42633
I20260812 16:57:06.605443 14208 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:06.606240 14208 heartbeater.cc:511] Master 127.8.195.190:42633 requested a full tablet report, sending...
I20260812 16:57:06.607867 13511 ts_manager.cc:194] Registered new tserver with Master: a36061dd81b54b2cb286e3739bf701ff (127.8.195.130:41895)
I20260812 16:57:06.609098 13511 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.130:34461
W20260812 16:57:06.947713 14216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:06.948035 14216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:06.948240 14216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:06.955875 14216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:06.956032 14216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:06.956241 14216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:57:06.965795 14216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:42633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:06.967227 14216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:06.969211 14216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:06.982807 14222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.982832 14232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:06.982899 14225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:06.986886 14216 server_base.cc:1034] running on GCE node
I20260812 16:57:06.987528 14216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:06.988524 14216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:06.989765 14216 hybrid_clock.cc:648] HybridClock initialized: now 1786553826989727 us; error 52 us; skew 500 ppm
I20260812 16:57:06.990101 14216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:06.992381 14216 webserver.cc:460] Webserver started at http://127.8.195.131:45277/ using document root <none> and password file <none>
I20260812 16:57:06.993078 14216 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:06.993245 14216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:06.993597 14216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:06.996510 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "20323f5141ff4278b87a0725999354f0"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "4810c309ef7d30a1624778755d6949f4"
server_key_iv: "f256ff222621cf5bf8c29fa120378b4c"
server_key_version: "encryptionkey@0"
I20260812 16:57:06.997476 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "20323f5141ff4278b87a0725999354f0"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "4810c309ef7d30a1624778755d6949f4"
server_key_iv: "f256ff222621cf5bf8c29fa120378b4c"
server_key_version: "encryptionkey@0"
I20260812 16:57:07.002702 14216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 16:57:07.006398 14248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:07.007828 14216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:57:07.008010 14216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "20323f5141ff4278b87a0725999354f0"
format_stamp: "Formatted at 2026-08-12 16:57:06 on dist-test-slave-z8x9"
server_key: "4810c309ef7d30a1624778755d6949f4"
server_key_iv: "f256ff222621cf5bf8c29fa120378b4c"
server_key_version: "encryptionkey@0"
I20260812 16:57:07.008292 14216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:07.022625 14216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:07.023500 14216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:07.023829 14216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:07.024868 14216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:07.026435 14216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:07.026566 14216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:07.026711 14216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:07.026810 14216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:07.061868 14216 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:38531
I20260812 16:57:07.061919 14463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:38531 every 8 connection(s)
I20260812 16:57:07.063184 14216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:57:07.065105 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14216
I20260812 16:57:07.065363 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:57:07.065922 8974 external_mini_cluster.cc:1428] Setting key 623ae923c5571a8b486d525f774363de
I20260812 16:57:07.076573 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.066147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:33233 (local address 127.8.195.190:42633)
0812 16:57:07.066406 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.066415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.068018 (+ 1603us) server_negotiation.cc:407] Connection header received
0812 16:57:07.069916 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.069920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.069990 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.070086 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:07.071478 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.072235 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.073893 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.074291 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.074432 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.075745 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:07.075751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:07.075754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:07.075772 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:07.075803 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:07.075816 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:07.075910 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.076135 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.076239 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 16:57:07.077651 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.066793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42633 (local address 127.8.195.131:33233)
0812 16:57:07.067853 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.067908 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.069637 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.070396 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.070404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.070492 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:07.071314 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.071333 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.072385 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.072391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.073737 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.073749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.074520 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.074529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.074764 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.075419 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:07.075422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:07.075460 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:07.075535 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:07.075912 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:07.075925 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:07.075935 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.076126 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.076543 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":361,"threads_started":1}
I20260812 16:57:07.078948 14468 heartbeater.cc:347] Connected to a master server at 127.8.195.190:42633
I20260812 16:57:07.079268 14468 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:07.080245 14468 heartbeater.cc:511] Master 127.8.195.190:42633 requested a full tablet report, sending...
I20260812 16:57:07.081816 13511 ts_manager.cc:194] Registered new tserver with Master: 20323f5141ff4278b87a0725999354f0 (127.8.195.131:38531)
I20260812 16:57:07.082592 13511 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.131:33233
I20260812 16:57:07.093667 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:57:07.103682 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.097839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.8.195.190:42633)
0812 16:57:07.098119 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.098125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.098423 (+ 298us) server_negotiation.cc:407] Connection header received
0812 16:57:07.098557 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.098561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.098632 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.098716 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:07.099180 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.099864 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.101241 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.101648 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.101741 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.102914 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:07.102919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:07.102923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:07.102939 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:07.102971 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:07.102983 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:07.103100 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.103345 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.103407 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 16:57:07.107606 13511 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:57:07.116040 13511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:57:07.138473 13933 heartbeater.cc:503] Master 127.8.195.190:42633 was elected leader, sending a full tablet report...
I20260812 16:57:07.141489 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.128553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:36679 (local address 127.0.0.1:59990)
0812 16:57:07.130942 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.131003 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.131211 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.133027 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.133035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.133072 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:07.133729 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.133755 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.135988 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.135996 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.138155 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.138173 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.140294 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.140301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.140621 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.140694 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.140824 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.141047 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2025,"mutex_wait_us":1,"thread_start_us":3672,"threads_started":1}
I20260812 16:57:07.141659 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.128995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:41895 (local address 127.0.0.1:50544)
0812 16:57:07.132542 (+ 3547us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.132566 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.132699 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.133632 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.133637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.133668 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:07.133983 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.133991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.135748 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.135755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.138404 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.138419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.140369 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.140376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.140972 (+ 596us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.141006 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.141091 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.141261 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3391,"mutex_wait_us":307,"thread_start_us":1426,"threads_started":1}
I20260812 16:57:07.141992 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.128847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.8.195.129:36679)
0812 16:57:07.131939 (+ 3092us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.131952 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.131989 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:57:07.132504 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.132516 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.132782 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.134013 (+ 1231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:07.134330 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.135844 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.138455 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.140072 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.141002 (+ 930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.141304 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.141427 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.141605 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2862,"thread_start_us":2730,"threads_started":1}
I20260812 16:57:07.142931 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.129568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.8.195.130:41895)
0812 16:57:07.132238 (+ 2670us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.132250 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.132842 (+ 592us) server_negotiation.cc:407] Connection header received
0812 16:57:07.132943 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.132951 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.133196 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.133389 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:07.134150 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.135576 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.138656 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.140115 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.140497 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.142327 (+ 1830us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.142401 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.142573 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2489,"thread_start_us":2378,"threads_started":1}
I20260812 16:57:07.144021 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.134335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38531 (local address 127.0.0.1:49806)
0812 16:57:07.134983 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.135003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.135116 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.135679 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.135685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.135719 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:07.136206 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.136222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.138223 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.138231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.140953 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.140966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.143334 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.143341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.143516 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.143547 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.143646 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.143788 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":338,"thread_start_us":410,"threads_started":1}
I20260812 16:57:07.145680 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.132987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.8.195.131:38531)
0812 16:57:07.134593 (+ 1606us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.134602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.135110 (+ 508us) server_negotiation.cc:407] Connection header received
0812 16:57:07.135259 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.135266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.135472 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.135651 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:07.136437 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.138042 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.141205 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.142723 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.142876 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.143148 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.145279 (+ 2131us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.145415 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1427,"thread_start_us":1468,"threads_started":1}
I20260812 16:57:07.146729 13790 tablet_service.cc:1467] Processing CreateTablet for tablet 5940da1d94d24259a506d197bcef7e1b (DEFAULT_TABLE table=test-table [id=f926e73fb52f43119a43e62249cfcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:07.146729 14070 tablet_service.cc:1467] Processing CreateTablet for tablet 5940da1d94d24259a506d197bcef7e1b (DEFAULT_TABLE table=test-table [id=f926e73fb52f43119a43e62249cfcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:07.148284 14070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940da1d94d24259a506d197bcef7e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:07.148892 13790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940da1d94d24259a506d197bcef7e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:07.149034 14334 tablet_service.cc:1467] Processing CreateTablet for tablet 5940da1d94d24259a506d197bcef7e1b (DEFAULT_TABLE table=test-table [id=f926e73fb52f43119a43e62249cfcf29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:07.150380 14334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5940da1d94d24259a506d197bcef7e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:07.162389 14500 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Bootstrap starting.
I20260812 16:57:07.165122 14506 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Bootstrap starting.
I20260812 16:57:07.165799 14500 tablet_bootstrap.cc:654] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:07.167245 14500 log.cc:824] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:07.169185 14506 tablet_bootstrap.cc:654] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:07.170673 14506 log.cc:824] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:07.170827 14500 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: No bootstrap required, opened a new log
I20260812 16:57:07.171157 14500 ts_tablet_manager.cc:1397] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 16:57:07.171921 14503 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Bootstrap starting.
I20260812 16:57:07.174618 14506 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: No bootstrap required, opened a new log
I20260812 16:57:07.174898 14506 ts_tablet_manager.cc:1397] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 16:57:07.175443 14503 tablet_bootstrap.cc:654] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:07.176685 14503 log.cc:824] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:07.183177 14500 raft_consensus.cc:348] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.183178 14506 raft_consensus.cc:348] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.183585 14500 raft_consensus.cc:374] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:07.183672 14506 raft_consensus.cc:374] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:07.183732 14500 raft_consensus.cc:729] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20323f5141ff4278b87a0725999354f0, State: Initialized, Role: FOLLOWER
I20260812 16:57:07.183802 14506 raft_consensus.cc:729] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa8d8d82f6f4387823f8017496d405e, State: Initialized, Role: FOLLOWER
I20260812 16:57:07.184175 14503 tablet_bootstrap.cc:492] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: No bootstrap required, opened a new log
I20260812 16:57:07.184475 14503 ts_tablet_manager.cc:1397] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20260812 16:57:07.184432 14500 consensus_queue.cc:260] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.184554 14506 consensus_queue.cc:260] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.185647 14468 heartbeater.cc:503] Master 127.8.195.190:42633 was elected leader, sending a full tablet report...
I20260812 16:57:07.187753 14500 ts_tablet_manager.cc:1428] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260812 16:57:07.189531 14506 ts_tablet_manager.cc:1428] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260812 16:57:07.194962 14503 raft_consensus.cc:348] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.195490 14503 raft_consensus.cc:374] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:07.195604 14503 raft_consensus.cc:729] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a36061dd81b54b2cb286e3739bf701ff, State: Initialized, Role: FOLLOWER
I20260812 16:57:07.196324 14503 consensus_queue.cc:260] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.197407 14208 heartbeater.cc:503] Master 127.8.195.190:42633 was elected leader, sending a full tablet report...
I20260812 16:57:07.198079 14503 ts_tablet_manager.cc:1428] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Time spent starting tablet: real 0.013s user 0.007s sys 0.008s
I20260812 16:57:07.218709 14516 raft_consensus.cc:482] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:57:07.219025 14516 raft_consensus.cc:504] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.220669 14516 leader_election.cc:288] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aa8d8d82f6f4387823f8017496d405e (127.8.195.129:36679), 20323f5141ff4278b87a0725999354f0 (127.8.195.131:38531)
I20260812 16:57:07.228226 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.221120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:36679 (local address 127.8.195.130:43159)
0812 16:57:07.221848 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.221865 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.222051 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.222477 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.222483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.222525 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:07.222965 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.222978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.224156 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.224162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.226069 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.226083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.227399 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.227409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.227624 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.227649 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.227725 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.227825 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":25,"thread_start_us":395,"threads_started":1}
I20260812 16:57:07.228283 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.221147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43159 (local address 127.8.195.129:36679)
0812 16:57:07.221373 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.221378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.221973 (+ 595us) server_negotiation.cc:407] Connection header received
0812 16:57:07.222197 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.222204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.222324 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.222426 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:07.223118 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.224026 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.226280 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.227170 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.227311 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.227364 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.227889 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.228029 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:57:07.228344 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.222146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:42737 (local address 127.8.195.131:38531)
0812 16:57:07.222378 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.222384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.222739 (+ 355us) server_negotiation.cc:407] Connection header received
0812 16:57:07.222883 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.222892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.222977 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.223080 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:07.223635 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.224543 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.226505 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.227323 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.227497 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.227570 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.227954 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.228085 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:57:07.228411 14520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.221939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:38531 (local address 127.8.195.130:42737)
0812 16:57:07.222627 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:07.222645 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:07.222746 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:07.223111 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:07.223115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:07.223138 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:07.223484 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.223496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.224724 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.224728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.226306 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:07.226321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.227530 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.227535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:07.227666 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.227688 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 16:57:07.227903 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 16:57:07.228034 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":3,"thread_start_us":356,"threads_started":1}
I20260812 16:57:07.229748 13826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940da1d94d24259a506d197bcef7e1b" candidate_uuid: "a36061dd81b54b2cb286e3739bf701ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa8d8d82f6f4387823f8017496d405e" is_pre_election: true
I20260812 16:57:07.229748 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940da1d94d24259a506d197bcef7e1b" candidate_uuid: "a36061dd81b54b2cb286e3739bf701ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20323f5141ff4278b87a0725999354f0" is_pre_election: true
I20260812 16:57:07.230393 14373 raft_consensus.cc:2454] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a36061dd81b54b2cb286e3739bf701ff in term 0.
I20260812 16:57:07.230393 13826 raft_consensus.cc:2454] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a36061dd81b54b2cb286e3739bf701ff in term 0.
I20260812 16:57:07.231128 13995 leader_election.cc:302] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa8d8d82f6f4387823f8017496d405e, a36061dd81b54b2cb286e3739bf701ff; no voters:
I20260812 16:57:07.231541 14516 raft_consensus.cc:2781] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:57:07.231710 14516 raft_consensus.cc:482] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:57:07.231885 14516 raft_consensus.cc:3037] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:07.234395 14516 raft_consensus.cc:504] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.235065 14516 leader_election.cc:288] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [CANDIDATE]: Term 1 election: Requested vote from peers 5aa8d8d82f6f4387823f8017496d405e (127.8.195.129:36679), 20323f5141ff4278b87a0725999354f0 (127.8.195.131:38531)
I20260812 16:57:07.235507 13826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940da1d94d24259a506d197bcef7e1b" candidate_uuid: "a36061dd81b54b2cb286e3739bf701ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa8d8d82f6f4387823f8017496d405e"
I20260812 16:57:07.235599 14373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5940da1d94d24259a506d197bcef7e1b" candidate_uuid: "a36061dd81b54b2cb286e3739bf701ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20323f5141ff4278b87a0725999354f0"
I20260812 16:57:07.235737 13826 raft_consensus.cc:3037] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:07.235862 14373 raft_consensus.cc:3037] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:07.238214 13826 raft_consensus.cc:2454] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a36061dd81b54b2cb286e3739bf701ff in term 1.
I20260812 16:57:07.238401 14373 raft_consensus.cc:2454] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a36061dd81b54b2cb286e3739bf701ff in term 1.
I20260812 16:57:07.238710 13995 leader_election.cc:302] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa8d8d82f6f4387823f8017496d405e, a36061dd81b54b2cb286e3739bf701ff; no voters:
I20260812 16:57:07.239015 14516 raft_consensus.cc:2781] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:07.239531 14516 raft_consensus.cc:686] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 1 LEADER]: Becoming Leader. State: Replica: a36061dd81b54b2cb286e3739bf701ff, State: Running, Role: LEADER
I20260812 16:57:07.240091 14516 consensus_queue.cc:237] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } }
I20260812 16:57:07.245051 13511 catalog_manager.cc:5495] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff reported cstate change: term changed from 0 to 1, leader changed from <none> to a36061dd81b54b2cb286e3739bf701ff (127.8.195.130). New cstate: current_term: 1 leader_uuid: "a36061dd81b54b2cb286e3739bf701ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a36061dd81b54b2cb286e3739bf701ff" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 41895 } health_report { overall_health: HEALTHY } } }
I20260812 16:57:07.269352 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.263215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.8.195.130:41895)
0812 16:57:07.263472 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.263479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.263594 (+ 115us) server_negotiation.cc:407] Connection header received
0812 16:57:07.263887 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.263892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.264003 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.264170 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:57:07.264598 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.265384 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.266895 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.267396 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.267535 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.268159 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:57:07.268656 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:57:07.268850 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.268978 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.269022 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 16:57:07.277611 14373 raft_consensus.cc:1261] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 1 FOLLOWER]: Refusing update from remote peer a36061dd81b54b2cb286e3739bf701ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:07.277658 13826 raft_consensus.cc:1261] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 1 FOLLOWER]: Refusing update from remote peer a36061dd81b54b2cb286e3739bf701ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:07.278694 14522 consensus_queue.cc:1035] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa8d8d82f6f4387823f8017496d405e" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 36679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:07.279071 14516 consensus_queue.cc:1035] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "20323f5141ff4278b87a0725999354f0" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:07.293121 14530 mvcc.cc:205] Tried to move back new op lower bound from 7317724476518801408 to 7317724476379316224. Current Snapshot: MvccSnapshot[applied={T|T < 7317724476518801408}]
I20260812 16:57:07.312357 13508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:07.312722 13508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:07.316421 13508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c52db08a30b4f36ab00d73ab8d75b55: Sending DeleteTablet for 3 replicas of tablet 5940da1d94d24259a506d197bcef7e1b
I20260812 16:57:07.317751 14070 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940da1d94d24259a506d197bcef7e1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:07 UTC) from {username='slave'} at 127.0.0.1:50544
I20260812 16:57:07.317703 14334 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940da1d94d24259a506d197bcef7e1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:07 UTC) from {username='slave'} at 127.0.0.1:49806
I20260812 16:57:07.317780 13790 tablet_service.cc:1514] Processing DeleteTablet for tablet 5940da1d94d24259a506d197bcef7e1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:07 UTC) from {username='slave'} at 127.0.0.1:59990
I20260812 16:57:07.324157 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13658
I20260812 16:57:07.328631 14548 tablet_replica.cc:330] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: stopping tablet replica
I20260812 16:57:07.329131 14548 raft_consensus.cc:2229] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:07.329772 14548 raft_consensus.cc:2258] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:07.332252 14548 ts_tablet_manager.cc:1905] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:07.332922 14547 tablet_replica.cc:330] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: stopping tablet replica
I20260812 16:57:07.333389 14547 raft_consensus.cc:2229] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:57:07.333783 14545 tablet_replica.cc:330] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: stopping tablet replica
I20260812 16:57:07.334227 14547 raft_consensus.cc:2258] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:07.334213 14545 raft_consensus.cc:2229] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:07.334708 14545 raft_consensus.cc:2258] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:07.336568 14547 ts_tablet_manager.cc:1905] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:07.340418 14545 ts_tablet_manager.cc:1905] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:07.341923 14548 ts_tablet_manager.cc:1918] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:07.342115 14548 log.cc:1196] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/5940da1d94d24259a506d197bcef7e1b
I20260812 16:57:07.342633 14548 ts_tablet_manager.cc:1939] T 5940da1d94d24259a506d197bcef7e1b P 5aa8d8d82f6f4387823f8017496d405e: Deleting consensus metadata
I20260812 16:57:07.345598 13481 catalog_manager.cc:4841] TS 5aa8d8d82f6f4387823f8017496d405e (127.8.195.129:36679): tablet 5940da1d94d24259a506d197bcef7e1b (table test-table [id=f926e73fb52f43119a43e62249cfcf29]) successfully deleted
I20260812 16:57:07.346798 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.326021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.8.195.129:36679)
0812 16:57:07.326264 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.326273 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.326988 (+ 715us) server_negotiation.cc:407] Connection header received
0812 16:57:07.327083 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.327091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.327254 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.327393 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:07.334179 (+ 6786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.335246 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.337209 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.337857 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.337954 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.344396 (+ 6442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:07.344408 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:07.344430 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:07.345088 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:07.345140 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:07.345162 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:07.345348 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.346301 (+ 953us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.346435 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:57:07.347834 14545 ts_tablet_manager.cc:1918] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:07.348117 14545 log.cc:1196] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/5940da1d94d24259a506d197bcef7e1b
I20260812 16:57:07.348387 13912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:07.348776 14545 ts_tablet_manager.cc:1939] T 5940da1d94d24259a506d197bcef7e1b P 20323f5141ff4278b87a0725999354f0: Deleting consensus metadata
I20260812 16:57:07.351356 14547 ts_tablet_manager.cc:1918] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:07.351629 14547 log.cc:1196] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/5940da1d94d24259a506d197bcef7e1b
I20260812 16:57:07.352137 13481 catalog_manager.cc:4841] TS 20323f5141ff4278b87a0725999354f0 (127.8.195.131:38531): tablet 5940da1d94d24259a506d197bcef7e1b (table test-table [id=f926e73fb52f43119a43e62249cfcf29]) successfully deleted
I20260812 16:57:07.352242 14547 ts_tablet_manager.cc:1939] T 5940da1d94d24259a506d197bcef7e1b P a36061dd81b54b2cb286e3739bf701ff: Deleting consensus metadata
I20260812 16:57:07.353838 13472 catalog_manager.cc:4841] TS a36061dd81b54b2cb286e3739bf701ff (127.8.195.130:41895): tablet 5940da1d94d24259a506d197bcef7e1b (table test-table [id=f926e73fb52f43119a43e62249cfcf29]) successfully deleted
I20260812 16:57:07.475721 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13658
I20260812 16:57:07.501412 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13948
I20260812 16:57:07.510253 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.502455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.8.195.130:41895)
0812 16:57:07.502654 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.502660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.502682 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:57:07.502753 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.502758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.502877 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.502990 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:07.503529 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.504678 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.506278 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.506980 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.507121 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.508445 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:07.508457 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:07.508477 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:07.509342 (+ 865us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:07.509406 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:07.509445 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:07.509560 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.509722 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.509832 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 16:57:07.511005 14193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:07.626663 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13948
I20260812 16:57:07.654549 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14216
I20260812 16:57:07.661345 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:07.655345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.8.195.131:38531)
0812 16:57:07.655554 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:07.655559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:07.655577 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:57:07.655748 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:07.655752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:07.655835 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:07.655920 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:07.656319 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.657104 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.658466 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:07.658840 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:07.658934 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:07.660000 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:07.660008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:07.660022 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:07.660605 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:07.660653 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:07.660673 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:07.660774 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:07.660895 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 16:57:07.660984 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 16:57:07.662073 14458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:07.753275 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14216
I20260812 16:57:07.780900 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 13439
I20260812 16:57:07.782056 13573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:07.893074 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 13439
2026-08-12T16:57:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2929 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:08 dist-test-slave-z8x9 krb5kdc[14565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:08 dist-test-slave-z8x9 krb5kdc[14565](info): set up 2 sockets
Aug 12 16:57:08 dist-test-slave-z8x9 krb5kdc[14565](info): commencing operation
krb5kdc: starting...
W20260812 16:57:09.347615 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:09 dist-test-slave-z8x9 krb5kdc[14565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:09.681346 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45593
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:45593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:10.040737 14595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:10.041042 14595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:10.041285 14595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:10.048885 14595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:10.049014 14595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:10.049119 14595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:10.049273 14595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:10.049329 14595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:10.059676 14595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:45593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:45593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:10.061124 14595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:10.062556 14595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:10.075839 14604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:10.076033 14603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:10.076846 14610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:10.079131 14595 server_base.cc:1034] running on GCE node
I20260812 16:57:10.080345 14595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:10.082582 14595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:10.083885 14595 hybrid_clock.cc:648] HybridClock initialized: now 1786553830083784 us; error 109 us; skew 500 ppm
I20260812 16:57:10.084405 14595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:10 dist-test-slave-z8x9 krb5kdc[14565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:10.089144 14595 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:10.091197 14595 webserver.cc:460] Webserver started at http://127.8.195.190:39061/ using document root <none> and password file <none>
I20260812 16:57:10.092118 14595 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:10.092259 14595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:10.092590 14595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:10.097631 14595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "186e0e7726c946d494159e8235399ff4"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "a6be78ef366e5da489c02bcd236abdd8"
server_key_iv: "b02931cc43aa96ba32de89971a68c7fa"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.098765 14595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "186e0e7726c946d494159e8235399ff4"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "a6be78ef366e5da489c02bcd236abdd8"
server_key_iv: "b02931cc43aa96ba32de89971a68c7fa"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.105957 14595 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260812 16:57:10.110049 14622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:10.111680 14595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:10.111891 14595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "186e0e7726c946d494159e8235399ff4"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "a6be78ef366e5da489c02bcd236abdd8"
server_key_iv: "b02931cc43aa96ba32de89971a68c7fa"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.112118 14595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:10.139235 14595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:10.144121 14595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:10.144459 14595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:10.166361 14595 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:45593
I20260812 16:57:10.166468 14726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:45593 every 8 connection(s)
I20260812 16:57:10.167886 14595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:10.173426 14728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:10.175727 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14595
I20260812 16:57:10.175946 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:10.176416 8974 external_mini_cluster.cc:1428] Setting key 8c9452c51c44778ea3ea01e7094097f2
I20260812 16:57:10.191838 14728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: Bootstrap starting.
I20260812 16:57:10.197383 14728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:10.199201 14728 log.cc:824] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:10.202643 14728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: No bootstrap required, opened a new log
Aug 12 16:57:10 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:10.211799 14728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } }
I20260812 16:57:10.212340 14728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:10.212486 14728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186e0e7726c946d494159e8235399ff4, State: Initialized, Role: FOLLOWER
I20260812 16:57:10.213317 14728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } }
I20260812 16:57:10.213618 14728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:10.213780 14728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:10.214004 14728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:10.216764 14728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } }
I20260812 16:57:10.217569 14728 leader_election.cc:302] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186e0e7726c946d494159e8235399ff4; no voters:
I20260812 16:57:10.218343 14728 leader_election.cc:288] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:10.218478 14739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:10.219189 14739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 186e0e7726c946d494159e8235399ff4, State: Running, Role: LEADER
I20260812 16:57:10.220069 14739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } }
I20260812 16:57:10.223521 14733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:10.178551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.8.195.190:45593)
0812 16:57:10.193779 (+ 15228us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:10.193796 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:10.193855 (+ 59us) server_negotiation.cc:407] Connection header received
0812 16:57:10.196040 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:10.196064 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:10.196471 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:10.196897 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:10.198627 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.200216 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.202683 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.203808 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.204008 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:10.210429 (+ 6421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:10.210447 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:10.210482 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:10.210566 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:10.215890 (+ 5324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:10.217020 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:10.217032 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:10.217044 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:10.217510 (+ 466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:10.218031 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:10.218038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:10.218041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:10.219137 (+ 1096us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:10.220919 (+ 1782us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:10.221319 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 16:57:10.221858 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14968,"thread_start_us":14697,"threads_started":1}
I20260812 16:57:10.223855 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186e0e7726c946d494159e8235399ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } } }
I20260812 16:57:10.224367 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:10.229005 14740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186e0e7726c946d494159e8235399ff4. Latest consensus state: current_term: 1 leader_uuid: "186e0e7726c946d494159e8235399ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186e0e7726c946d494159e8235399ff4" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 45593 } } }
I20260812 16:57:10.229521 14740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:10.230767 14728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:10.235175 14748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:10.243058 14748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:10.256426 14748 catalog_manager.cc:1347] Generated new cluster ID: 19ff47f3f70844869d0c97552374d26d
I20260812 16:57:10.256593 14748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:10.293820 14748 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:10.295799 14748 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:10.308007 14748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: Generated new TSK 0
I20260812 16:57:10.309144 14748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.195.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
Entry for principal kudu/127.8.195.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab.
I20260812 16:57:10.384892 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:10.738736 14787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:10.739045 14787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:10.739204 14787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:10.746868 14787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:10.746995 14787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:10.747226 14787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:57:10.757539 14787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.129.keytab
--principal=kudu/127.8.195.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:10.758903 14787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:10.760239 14787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:10.773900 14801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:10.773939 14804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:10.773993 14799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:10.778815 14787 server_base.cc:1034] running on GCE node
I20260812 16:57:10.779631 14787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:10.780737 14787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:10.781996 14787 hybrid_clock.cc:648] HybridClock initialized: now 1786553830781952 us; error 40 us; skew 500 ppm
I20260812 16:57:10.782356 14787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:10 dist-test-slave-z8x9 krb5kdc[14565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:10.785658 14787 init.cc:377] Logged in from keytab as kudu/127.8.195.129@KRBTEST.COM (short username kudu)
I20260812 16:57:10.787360 14787 webserver.cc:460] Webserver started at http://127.8.195.129:37579/ using document root <none> and password file <none>
I20260812 16:57:10.788090 14787 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:10.788242 14787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:10.788580 14787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:10.791641 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "946c4f546592497ba4f8889041548427"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "286d4bd283ae9b87ada9fd2d76caab18"
server_key_iv: "6698243fbdd06c834f2fd9c6aa2f602f"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.792459 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "946c4f546592497ba4f8889041548427"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "286d4bd283ae9b87ada9fd2d76caab18"
server_key_iv: "6698243fbdd06c834f2fd9c6aa2f602f"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.797691 14787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260812 16:57:10.801313 14821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:10.802600 14787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:10.802796 14787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "946c4f546592497ba4f8889041548427"
format_stamp: "Formatted at 2026-08-12 16:57:10 on dist-test-slave-z8x9"
server_key: "286d4bd283ae9b87ada9fd2d76caab18"
server_key_iv: "6698243fbdd06c834f2fd9c6aa2f602f"
server_key_version: "encryptionkey@0"
I20260812 16:57:10.803011 14787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:10.821683 14787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:10.825908 14787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:10.826227 14787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:10.827229 14787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:10.828663 14787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:10.828773 14787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:10.828934 14787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:10.829018 14787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:10.864005 14787 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:33845
I20260812 16:57:10.864053 15082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:33845 every 8 connection(s)
I20260812 16:57:10.865353 14787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:57:10.870426 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 14787
I20260812 16:57:10.870615 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:57:10.871169 8974 external_mini_cluster.cc:1428] Setting key 024761f8a984b1ad8783d7075ce08132
Aug 12 16:57:10 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.129@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:10.889524 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:10.867947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:34973 (local address 127.8.195.190:45593)
0812 16:57:10.868413 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:10.868417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:10.869478 (+ 1061us) server_negotiation.cc:407] Connection header received
0812 16:57:10.871996 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:10.872002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:10.872095 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:10.872213 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:10.874062 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.874827 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.877434 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.877845 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.877947 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:10.882623 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:10.882629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:10.882632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:10.882667 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:10.885624 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:10.886712 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:10.886716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:10.886719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:10.886847 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:10.887393 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:10.887397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:10.887399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:10.887884 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:10.888052 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:10.888946 (+ 894us) server_negotiation.cc:299] Negotiation successful
0812 16:57:10.889223 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":258,"threads_started":1}
I20260812 16:57:10.890635 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:10.868528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45593 (local address 127.8.195.129:34973)
0812 16:57:10.869313 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:10.869353 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:10.871660 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:10.872471 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:10.872480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:10.873014 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:10.873869 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:10.873893 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.874999 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.875008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:10.877281 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:10.877299 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:10.878084 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:10.878094 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:10.878427 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:10.879202 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:10.879244 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:10.882362 (+ 3118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:10.885795 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:10.885807 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:10.885829 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:10.886579 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:10.886960 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:10.886968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:10.886974 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:10.887274 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:10.888039 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:10.888045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:10.888489 (+ 444us) client_negotiation.cc:769] Sending connection context
0812 16:57:10.888778 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 16:57:10.889306 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":282,"threads_started":1}
I20260812 16:57:10.892357 15084 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45593
I20260812 16:57:10.892697 15084 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:10.893683 15084 heartbeater.cc:511] Master 127.8.195.190:45593 requested a full tablet report, sending...
I20260812 16:57:10.896018 14661 ts_manager.cc:194] Registered new tserver with Master: 946c4f546592497ba4f8889041548427 (127.8.195.129:33845)
I20260812 16:57:10.897583 14661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.129@KRBTEST.COM'} at 127.8.195.129:34973
WARNING: no policy specified for kudu/127.8.195.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
Entry for principal kudu/127.8.195.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab.
I20260812 16:57:10.948257 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:11.304489 15094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:11.304788 15094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:11.304944 15094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:11.312517 15094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:11.312638 15094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:11.312862 15094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:57:11.323015 15094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.130.keytab
--principal=kudu/127.8.195.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:11.324337 15094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:11.325700 15094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:11.339119 15103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:11.339190 15109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:11.339206 15111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:11.346907 15094 server_base.cc:1034] running on GCE node
I20260812 16:57:11.347606 15094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:11.348640 15094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:11.349867 15094 hybrid_clock.cc:648] HybridClock initialized: now 1786553831349824 us; error 31 us; skew 500 ppm
I20260812 16:57:11.350203 15094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:11 dist-test-slave-z8x9 krb5kdc[14565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553831, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:11.353695 15094 init.cc:377] Logged in from keytab as kudu/127.8.195.130@KRBTEST.COM (short username kudu)
I20260812 16:57:11.355312 15094 webserver.cc:460] Webserver started at http://127.8.195.130:39707/ using document root <none> and password file <none>
I20260812 16:57:11.356009 15094 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:11.356112 15094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:11.356379 15094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:11.359362 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "2872b4db7e784b6785c62f17731931f2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "ee53beb75f2d289cd510d7796fa68e9c"
server_key_iv: "e674cd3573e98b1039ddbf72e11f27dc"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.360140 15094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "2872b4db7e784b6785c62f17731931f2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "ee53beb75f2d289cd510d7796fa68e9c"
server_key_iv: "e674cd3573e98b1039ddbf72e11f27dc"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.365792 15094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 16:57:11.369426 15127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.370824 15094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:11.370952 15094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "2872b4db7e784b6785c62f17731931f2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "ee53beb75f2d289cd510d7796fa68e9c"
server_key_iv: "e674cd3573e98b1039ddbf72e11f27dc"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.371119 15094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:11.384209 15094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:11.388527 15094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:11.388804 15094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:11.389844 15094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:11.391278 15094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:11.391360 15094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.391444 15094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:11.391530 15094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.426869 15094 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:34753
I20260812 16:57:11.426918 15355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:34753 every 8 connection(s)
I20260812 16:57:11.428252 15094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:57:11.432099 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15094
I20260812 16:57:11.432307 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:57:11.432821 8974 external_mini_cluster.cc:1428] Setting key c479949d750702b6ff3afd53458ca4b6
Aug 12 16:57:11 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553831, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.130@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:11.452209 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:11.430973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43569 (local address 127.8.195.190:45593)
0812 16:57:11.431467 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:11.431472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:11.432522 (+ 1050us) server_negotiation.cc:407] Connection header received
0812 16:57:11.434955 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:11.434979 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:11.435063 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:11.435181 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:11.436685 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:11.437967 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:11.439682 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:11.440443 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:11.440598 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:11.444328 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:11.444338 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:11.444345 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:11.444393 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:11.448370 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:11.449394 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:11.449403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:11.449409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:11.449557 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:11.450135 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:11.450139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:11.450141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:11.450412 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:11.450535 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:11.451415 (+ 880us) server_negotiation.cc:299] Negotiation successful
0812 16:57:11.451684 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":263,"threads_started":1}
I20260812 16:57:11.452837 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:11.431420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45593 (local address 127.8.195.130:43569)
0812 16:57:11.432341 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:11.432398 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:11.434666 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:11.435387 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:11.435396 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:11.435939 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:11.436495 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:11.436513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:11.438188 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:11.438193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:11.439522 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:11.439535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:11.440688 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:11.440694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:11.440903 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:11.441714 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:11.441739 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:11.444064 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:11.448548 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:11.448560 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:11.448579 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:11.449221 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:11.449687 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:11.449693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:11.449699 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:11.449990 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:11.450547 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:11.450559 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:11.451011 (+ 452us) client_negotiation.cc:769] Sending connection context
0812 16:57:11.451242 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 16:57:11.451647 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":359,"threads_started":1}
I20260812 16:57:11.454484 15356 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45593
I20260812 16:57:11.454818 15356 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:11.455549 15356 heartbeater.cc:511] Master 127.8.195.190:45593 requested a full tablet report, sending...
I20260812 16:57:11.457123 14661 ts_manager.cc:194] Registered new tserver with Master: 2872b4db7e784b6785c62f17731931f2 (127.8.195.130:34753)
I20260812 16:57:11.458510 14661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.130@KRBTEST.COM'} at 127.8.195.130:43569
WARNING: no policy specified for kudu/127.8.195.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
Entry for principal kudu/127.8.195.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab.
I20260812 16:57:11.508119 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:37825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:11.862962 15375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:11.863317 15375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:11.863524 15375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:11.871196 15375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:11.871318 15375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:11.871495 15375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:57:11.881944 15375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:37825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.131.keytab
--principal=kudu/127.8.195.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:45593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:11.883327 15375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:11.884675 15375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:11.898365 15386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:11.898317 15389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:11.898360 15393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:11.899180 15375 server_base.cc:1034] running on GCE node
I20260812 16:57:11.899786 15375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:11.900874 15375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:11.901448 15084 heartbeater.cc:503] Master 127.8.195.190:45593 was elected leader, sending a full tablet report...
I20260812 16:57:11.902064 15375 hybrid_clock.cc:648] HybridClock initialized: now 1786553831902028 us; error 33 us; skew 500 ppm
I20260812 16:57:11.902505 15375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:11 dist-test-slave-z8x9 krb5kdc[14565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553831, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:11.906075 15375 init.cc:377] Logged in from keytab as kudu/127.8.195.131@KRBTEST.COM (short username kudu)
I20260812 16:57:11.907833 15375 webserver.cc:460] Webserver started at http://127.8.195.131:41689/ using document root <none> and password file <none>
I20260812 16:57:11.908548 15375 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:11.908655 15375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:11.908924 15375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:11.912621 15375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "4b11669a5cc142148edc2379fb2279b2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "36747e43ee459e3cfc54521c3fd32351"
server_key_iv: "043bc4a416cd18c8ba0440122a5f4bcd"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.913475 15375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "4b11669a5cc142148edc2379fb2279b2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "36747e43ee459e3cfc54521c3fd32351"
server_key_iv: "043bc4a416cd18c8ba0440122a5f4bcd"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.918774 15375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:57:11.922133 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.923477 15375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:57:11.923643 15375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "4b11669a5cc142148edc2379fb2279b2"
format_stamp: "Formatted at 2026-08-12 16:57:11 on dist-test-slave-z8x9"
server_key: "36747e43ee459e3cfc54521c3fd32351"
server_key_iv: "043bc4a416cd18c8ba0440122a5f4bcd"
server_key_version: "encryptionkey@0"
I20260812 16:57:11.923857 15375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:11.951242 15375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:11.956030 15375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:11.956360 15375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:11.957465 15375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:11.958910 15375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:11.959017 15375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.959173 15375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:11.959251 15375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:11.993752 15375 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:44697
I20260812 16:57:11.993810 15624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:44697 every 8 connection(s)
I20260812 16:57:11.995087 15375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:57:12.002557 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15375
I20260812 16:57:12.002777 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:57:12.003297 8974 external_mini_cluster.cc:1428] Setting key 1c5e5469c46fb416d67e783615f9097b
Aug 12 16:57:12 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553831, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.131@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:12.015949 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:11.997804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:44063 (local address 127.8.195.190:45593)
0812 16:57:11.998289 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:11.998295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:11.999290 (+ 995us) server_negotiation.cc:407] Connection header received
0812 16:57:12.001118 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.001123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.001257 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.001357 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:12.003005 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.004236 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.006333 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.006828 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.006950 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.010604 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:12.010609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:12.010613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:12.010663 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:12.013296 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.013989 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.013993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.013995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.014112 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.014491 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.014494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.014496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.014753 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:12.014876 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.015501 (+ 625us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.015664 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":211,"threads_started":1}
I20260812 16:57:12.017042 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:11.998429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:45593 (local address 127.8.195.131:44063)
0812 16:57:11.999123 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:11.999167 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.000843 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.001584 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.001592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.002117 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:12.002787 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.002814 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.004415 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.004421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.006185 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.006200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.007073 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.007078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.007284 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.008071 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:57:12.008095 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:12.010395 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:12.013420 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:12.013427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:12.013440 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:12.013886 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:12.014204 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:57:12.014208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:57:12.014210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:57:12.014398 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:57:12.014865 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:12.014871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:12.015185 (+ 314us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.015426 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.015820 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":273,"threads_started":1}
I20260812 16:57:12.018594 15627 heartbeater.cc:347] Connected to a master server at 127.8.195.190:45593
I20260812 16:57:12.018945 15627 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:12.019649 15627 heartbeater.cc:511] Master 127.8.195.190:45593 requested a full tablet report, sending...
I20260812 16:57:12.021087 14661 ts_manager.cc:194] Registered new tserver with Master: 4b11669a5cc142148edc2379fb2279b2 (127.8.195.131:44697)
I20260812 16:57:12.021910 14661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.195.131@KRBTEST.COM'} at 127.8.195.131:44063
I20260812 16:57:12.031163 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:57:12.048269 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.035522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33684 (local address 127.8.195.190:45593)
0812 16:57:12.035712 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.035721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.036089 (+ 368us) server_negotiation.cc:407] Connection header received
0812 16:57:12.036289 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.036294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.036374 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.036478 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:12.037958 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.038699 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.040016 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.040428 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.040569 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.042976 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:12.042983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:12.042989 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:12.043027 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:12.045567 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.046228 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.046233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.046237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.046363 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.046744 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.046750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.046753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.047086 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:12.047359 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.047710 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.047953 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:57:12.052290 14661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:57:12.059221 14661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:57:12.096213 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.072351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.8.195.129:33845)
0812 16:57:12.086514 (+ 14163us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.086527 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.086568 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:57:12.086704 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.086715 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.087075 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.087295 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:12.088540 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.090300 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.093362 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.094897 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.095065 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.095312 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.095680 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.095854 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13909,"thread_start_us":15920,"threads_started":1}
I20260812 16:57:12.096246 15654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.072012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33845 (local address 127.0.0.1:54226)
0812 16:57:12.083840 (+ 11828us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:12.083903 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.084179 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.087637 (+ 3458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.087645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.087682 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:12.088123 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.088137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.090466 (+ 2329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.090472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.093113 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.093133 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.095168 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.095175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.095428 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.095470 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.095621 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.095833 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11434,"thread_start_us":17922,"threads_started":1}
I20260812 16:57:12.100260 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.072165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:44697 (local address 127.0.0.1:57518)
0812 16:57:12.090205 (+ 18040us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:12.090239 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.090462 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.092823 (+ 2361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.092828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.092852 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:12.093362 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.093375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.095082 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.095087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.097985 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.097998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.099383 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.099391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.099579 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.099613 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.099712 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.099847 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17918,"mutex_wait_us":1,"thread_start_us":11071,"threads_started":1}
I20260812 16:57:12.100301 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.072659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.8.195.131:44697)
0812 16:57:12.089611 (+ 16952us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.089619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.090348 (+ 729us) server_negotiation.cc:407] Connection header received
0812 16:57:12.090682 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.090691 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.092319 (+ 1628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.092490 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:12.093572 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.094914 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.098153 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.099080 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.099248 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.099491 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.099772 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.099945 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16772,"thread_start_us":16943,"threads_started":1}
I20260812 16:57:12.100494 14941 tablet_service.cc:1467] Processing CreateTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 (DEFAULT_TABLE table=test-table [id=6f1a4365b0924aceb2ff647c2e321712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:12.102305 14941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b8e12048cc45fe828f7e486a471cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:12.102505 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.074029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:34753 (local address 127.0.0.1:36320)
0812 16:57:12.093102 (+ 19073us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:12.093122 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.093491 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.094531 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.094537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.094566 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:12.094895 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.094906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.097249 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.097254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.099443 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.099461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.101310 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.101317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.101891 (+ 574us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.101926 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.102016 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.102139 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17572,"thread_start_us":17472,"threads_started":1}
I20260812 16:57:12.103228 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.074581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.8.195.130:34753)
0812 16:57:12.093959 (+ 19378us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.093967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.093993 (+ 26us) server_negotiation.cc:407] Connection header received
0812 16:57:12.094079 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.094087 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.094323 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.094511 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:12.095092 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.097050 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.099643 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.101063 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.101264 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.101453 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.102853 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.102994 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19221,"thread_start_us":19060,"threads_started":1}
I20260812 16:57:12.104494 15499 tablet_service.cc:1467] Processing CreateTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 (DEFAULT_TABLE table=test-table [id=6f1a4365b0924aceb2ff647c2e321712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:12.106052 15499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b8e12048cc45fe828f7e486a471cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:12.106175 15218 tablet_service.cc:1467] Processing CreateTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 (DEFAULT_TABLE table=test-table [id=6f1a4365b0924aceb2ff647c2e321712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:12.107582 15218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b8e12048cc45fe828f7e486a471cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:12.118361 15663 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Bootstrap starting.
I20260812 16:57:12.122709 15664 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Bootstrap starting.
I20260812 16:57:12.123065 15663 tablet_bootstrap.cc:654] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:12.124560 15663 log.cc:824] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:12.127645 15664 tablet_bootstrap.cc:654] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:12.128098 15665 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Bootstrap starting.
I20260812 16:57:12.129022 15664 log.cc:824] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:12.132340 15665 tablet_bootstrap.cc:654] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:12.132609 15664 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: No bootstrap required, opened a new log
I20260812 16:57:12.132622 15663 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: No bootstrap required, opened a new log
I20260812 16:57:12.132932 15664 ts_tablet_manager.cc:1397] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 16:57:12.132952 15663 ts_tablet_manager.cc:1397] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260812 16:57:12.133770 15665 log.cc:824] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:12.141134 15663 raft_consensus.cc:348] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.141644 15663 raft_consensus.cc:374] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:12.141747 15663 raft_consensus.cc:729] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946c4f546592497ba4f8889041548427, State: Initialized, Role: FOLLOWER
I20260812 16:57:12.142499 15663 consensus_queue.cc:260] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.144451 15663 ts_tablet_manager.cc:1428] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260812 16:57:12.145099 15665 tablet_bootstrap.cc:492] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: No bootstrap required, opened a new log
I20260812 16:57:12.145442 15665 ts_tablet_manager.cc:1397] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.007s
I20260812 16:57:12.145884 15664 raft_consensus.cc:348] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.146279 15664 raft_consensus.cc:374] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:12.146401 15664 raft_consensus.cc:729] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2872b4db7e784b6785c62f17731931f2, State: Initialized, Role: FOLLOWER
I20260812 16:57:12.147027 15664 consensus_queue.cc:260] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.148212 15356 heartbeater.cc:503] Master 127.8.195.190:45593 was elected leader, sending a full tablet report...
I20260812 16:57:12.148926 15664 ts_tablet_manager.cc:1428] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260812 16:57:12.155459 15665 raft_consensus.cc:348] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.155859 15665 raft_consensus.cc:374] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:12.155936 15665 raft_consensus.cc:729] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b11669a5cc142148edc2379fb2279b2, State: Initialized, Role: FOLLOWER
I20260812 16:57:12.156538 15665 consensus_queue.cc:260] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.157612 15627 heartbeater.cc:503] Master 127.8.195.190:45593 was elected leader, sending a full tablet report...
I20260812 16:57:12.158131 15665 ts_tablet_manager.cc:1428] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260812 16:57:12.163767 15679 raft_consensus.cc:482] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:57:12.164052 15679 raft_consensus.cc:504] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.166023 15679 leader_election.cc:288] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946c4f546592497ba4f8889041548427 (127.8.195.129:33845), 2872b4db7e784b6785c62f17731931f2 (127.8.195.130:34753)
I20260812 16:57:12.173250 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.166289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:33845 (local address 127.8.195.131:45041)
0812 16:57:12.166511 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:12.166533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.166664 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.167707 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.167712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.167736 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:12.167925 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.167940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.169360 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.169366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.171123 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.171136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.172453 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.172459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.172645 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.172680 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.172774 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.172906 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":85}
I20260812 16:57:12.173246 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.167192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:45041 (local address 127.8.195.129:33845)
0812 16:57:12.167408 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.167414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.167433 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:57:12.167501 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.167506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.167576 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.167667 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:12.168071 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.169218 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.171290 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.172278 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.172454 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.172528 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.172823 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.172965 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:57:12.174558 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.167081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:60761 (local address 127.8.195.130:34753)
0812 16:57:12.167433 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.167441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.167461 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:57:12.167528 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.167533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.167599 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.167699 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:12.168582 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.169416 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.171917 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.173065 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.173246 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.173669 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.174165 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.174297 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260812 16:57:12.174345 14982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b8e12048cc45fe828f7e486a471cb8" candidate_uuid: "4b11669a5cc142148edc2379fb2279b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c4f546592497ba4f8889041548427" is_pre_election: true
I20260812 16:57:12.174832 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.166510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:34753 (local address 127.8.195.131:60761)
0812 16:57:12.167248 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:12.167266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:12.167394 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:12.168105 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:12.168110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:12.168140 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:12.168416 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.168422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.169572 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.169578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.171736 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:12.171752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.173874 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.173881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:12.174001 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.174024 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 16:57:12.174136 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 16:57:12.174259 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":3,"thread_start_us":1339,"threads_started":1}
I20260812 16:57:12.174925 14982 raft_consensus.cc:2454] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b11669a5cc142148edc2379fb2279b2 in term 0.
I20260812 16:57:12.175642 15415 leader_election.cc:302] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b11669a5cc142148edc2379fb2279b2, 946c4f546592497ba4f8889041548427; no voters:
I20260812 16:57:12.176120 15679 raft_consensus.cc:2781] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:57:12.176286 15679 raft_consensus.cc:482] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:57:12.176167 15279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b8e12048cc45fe828f7e486a471cb8" candidate_uuid: "4b11669a5cc142148edc2379fb2279b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2872b4db7e784b6785c62f17731931f2" is_pre_election: true
I20260812 16:57:12.176388 15679 raft_consensus.cc:3037] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:12.176676 15279 raft_consensus.cc:2454] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b11669a5cc142148edc2379fb2279b2 in term 0.
I20260812 16:57:12.179630 15679 raft_consensus.cc:504] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.180310 15679 leader_election.cc:288] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [CANDIDATE]: Term 1 election: Requested vote from peers 946c4f546592497ba4f8889041548427 (127.8.195.129:33845), 2872b4db7e784b6785c62f17731931f2 (127.8.195.130:34753)
W20260812 16:57:12.180409 15363 tablet.cc:2369] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:57:12.180794 14982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b8e12048cc45fe828f7e486a471cb8" candidate_uuid: "4b11669a5cc142148edc2379fb2279b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946c4f546592497ba4f8889041548427"
I20260812 16:57:12.181005 14982 raft_consensus.cc:3037] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:12.181746 15279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b8e12048cc45fe828f7e486a471cb8" candidate_uuid: "4b11669a5cc142148edc2379fb2279b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2872b4db7e784b6785c62f17731931f2"
I20260812 16:57:12.181955 15279 raft_consensus.cc:3037] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:12.184237 14982 raft_consensus.cc:2454] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b11669a5cc142148edc2379fb2279b2 in term 1.
I20260812 16:57:12.184302 15279 raft_consensus.cc:2454] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b11669a5cc142148edc2379fb2279b2 in term 1.
I20260812 16:57:12.184765 15413 leader_election.cc:302] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2872b4db7e784b6785c62f17731931f2, 4b11669a5cc142148edc2379fb2279b2; no voters:
I20260812 16:57:12.185249 15679 raft_consensus.cc:2781] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:12.185755 15679 raft_consensus.cc:686] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 1 LEADER]: Becoming Leader. State: Replica: 4b11669a5cc142148edc2379fb2279b2, State: Running, Role: LEADER
I20260812 16:57:12.186236 15679 consensus_queue.cc:237] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } }
I20260812 16:57:12.191125 14657 catalog_manager.cc:5495] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b11669a5cc142148edc2379fb2279b2 (127.8.195.131). New cstate: current_term: 1 leader_uuid: "4b11669a5cc142148edc2379fb2279b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b11669a5cc142148edc2379fb2279b2" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260812 16:57:12.221625 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.216037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.8.195.131:44697)
0812 16:57:12.216215 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.216219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.216237 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:57:12.216498 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.216502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.216586 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.216716 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:57:12.217113 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.218032 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.219422 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.219818 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.219914 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.220465 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:57:12.220976 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:57:12.221087 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.221350 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.221422 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:57:12.230365 15279 raft_consensus.cc:1261] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 1 FOLLOWER]: Refusing update from remote peer 4b11669a5cc142148edc2379fb2279b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:12.230365 14982 raft_consensus.cc:1261] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 1 FOLLOWER]: Refusing update from remote peer 4b11669a5cc142148edc2379fb2279b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:12.231580 15681 consensus_queue.cc:1035] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2872b4db7e784b6785c62f17731931f2" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:12.232048 15679 consensus_queue.cc:1035] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "946c4f546592497ba4f8889041548427" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 33845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:12.262233 14661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:12.262564 14661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:12.266541 14661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 186e0e7726c946d494159e8235399ff4: Sending DeleteTablet for 3 replicas of tablet 37b8e12048cc45fe828f7e486a471cb8
I20260812 16:57:12.267879 15499 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:12 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:57518
I20260812 16:57:12.267966 14941 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:12 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:54226
I20260812 16:57:12.268889 15701 tablet_replica.cc:330] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: stopping tablet replica
I20260812 16:57:12.268980 15218 tablet_service.cc:1514] Processing DeleteTablet for tablet 37b8e12048cc45fe828f7e486a471cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:12 UTC) from {username='kudu', principal='kudu/127.8.195.190@KRBTEST.COM'} at 127.0.0.1:36320
I20260812 16:57:12.269556 15701 raft_consensus.cc:2229] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:57:12.270159 15702 tablet_replica.cc:330] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: stopping tablet replica
I20260812 16:57:12.270160 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14787
I20260812 16:57:12.270478 15701 raft_consensus.cc:2258] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:12.270598 15702 raft_consensus.cc:2229] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:12.271122 15702 raft_consensus.cc:2258] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:12.273622 15702 ts_tablet_manager.cc:1905] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:12.273988 15701 ts_tablet_manager.cc:1905] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:12.281651 15701 ts_tablet_manager.cc:1918] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:12.281965 15701 log.cc:1196] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/37b8e12048cc45fe828f7e486a471cb8
I20260812 16:57:12.282262 15702 ts_tablet_manager.cc:1918] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:12.282440 15702 log.cc:1196] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-0/wal/wals/37b8e12048cc45fe828f7e486a471cb8
I20260812 16:57:12.282886 15701 ts_tablet_manager.cc:1939] T 37b8e12048cc45fe828f7e486a471cb8 P 4b11669a5cc142148edc2379fb2279b2: Deleting consensus metadata
I20260812 16:57:12.282998 15702 ts_tablet_manager.cc:1939] T 37b8e12048cc45fe828f7e486a471cb8 P 946c4f546592497ba4f8889041548427: Deleting consensus metadata
I20260812 16:57:12.284880 14628 catalog_manager.cc:4841] TS 4b11669a5cc142148edc2379fb2279b2 (127.8.195.131:44697): tablet 37b8e12048cc45fe828f7e486a471cb8 (table test-table [id=6f1a4365b0924aceb2ff647c2e321712]) successfully deleted
I20260812 16:57:12.284951 14629 catalog_manager.cc:4841] TS 946c4f546592497ba4f8889041548427 (127.8.195.129:33845): tablet 37b8e12048cc45fe828f7e486a471cb8 (table test-table [id=6f1a4365b0924aceb2ff647c2e321712]) successfully deleted
I20260812 16:57:12.289144 15703 tablet_replica.cc:330] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: stopping tablet replica
I20260812 16:57:12.289717 15703 raft_consensus.cc:2229] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:12.290263 15703 raft_consensus.cc:2258] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:12.292578 15703 ts_tablet_manager.cc:1905] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 16:57:12 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.129@KRBTEST.COM
I20260812 16:57:12.299880 15703 ts_tablet_manager.cc:1918] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:12.300066 15703 log.cc:1196] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/37b8e12048cc45fe828f7e486a471cb8
I20260812 16:57:12.300503 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.272069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54242 (local address 127.8.195.129:33845)
0812 16:57:12.272238 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.272245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.276154 (+ 3909us) server_negotiation.cc:407] Connection header received
0812 16:57:12.285187 (+ 9033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.285192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.285294 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.285377 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:12.286815 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.287608 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.288908 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.289466 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.289605 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.294270 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:12.294283 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:12.294297 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:12.294336 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:12.297421 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.298135 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.298151 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.298154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.298293 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.298708 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.298714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.298719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.299066 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:12.299387 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.299701 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.299926 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:57:12.300659 15703 ts_tablet_manager.cc:1939] T 37b8e12048cc45fe828f7e486a471cb8 P 2872b4db7e784b6785c62f17731931f2: Deleting consensus metadata
I20260812 16:57:12.301268 15071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:12.302619 14626 catalog_manager.cc:4841] TS 2872b4db7e784b6785c62f17731931f2 (127.8.195.130:34753): tablet 37b8e12048cc45fe828f7e486a471cb8 (table test-table [id=6f1a4365b0924aceb2ff647c2e321712]) successfully deleted
I20260812 16:57:12.393311 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14787
I20260812 16:57:12.431214 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15094
Aug 12 16:57:12 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.130@KRBTEST.COM
I20260812 16:57:12.453928 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.432388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.8.195.130:34753)
0812 16:57:12.432555 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.432559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.432577 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:57:12.432628 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.432631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.432692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.432776 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:12.434571 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.435806 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.437871 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.438496 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.438616 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.447457 (+ 8841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:12.447469 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:12.447484 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:12.447522 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:12.450566 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.451464 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.451472 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.451475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.451607 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.452021 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.452024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.452026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.452594 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:12.452830 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.453262 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.453499 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 16:57:12.454794 15346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:12.545450 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15094
I20260812 16:57:12.580235 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15375
Aug 12 16:57:12 dist-test-slave-z8x9 krb5kdc[14565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.131@KRBTEST.COM
I20260812 16:57:12.599435 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:12.581243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.8.195.131:44697)
0812 16:57:12.581429 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:12.581433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:12.581452 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:57:12.581635 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:12.581639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:12.581713 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:12.581791 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:12.583701 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.584526 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.586528 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:12.586971 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:12.587076 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:12.592955 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:12.592968 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:12.592983 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:12.593024 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:12.596383 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.597318 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.597325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.597328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.597482 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:12.597935 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:12.597938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:12.597940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:12.598303 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:12.598558 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:12.598950 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 16:57:12.599117 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:57:12.600195 15617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:12.728186 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15375
I20260812 16:57:12.757830 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 14595
I20260812 16:57:12.758987 14721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:12.870721 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 14595
2026-08-12T16:57:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4975 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T16:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:12Z Disabled control of system clock
I20260812 16:57:12.975526 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34399
--webserver_interface=127.8.195.190
--webserver_port=0
--builtin_ntp_servers=127.8.195.148:34925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:34399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:57:13.330854 15721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:13.331193 15721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:13.331373 15721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:13.339097 15721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:13.339236 15721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:13.339334 15721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:13.339450 15721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:13.339532 15721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:13.349237 15721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:34399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:34399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:13.350633 15721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:13.352501 15721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:13.365247 15740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:13.365310 15733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:13.365283 15735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:13.366245 15721 server_base.cc:1034] running on GCE node
I20260812 16:57:13.367116 15721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:13.368549 15721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:13.369755 15721 hybrid_clock.cc:648] HybridClock initialized: now 1786553833369709 us; error 37 us; skew 500 ppm
I20260812 16:57:13.370131 15721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:13.372463 15721 webserver.cc:460] Webserver started at http://127.8.195.190:46283/ using document root <none> and password file <none>
I20260812 16:57:13.373216 15721 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:13.373334 15721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:13.373682 15721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:13.376618 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "8db20acf55194c63bcbfcf58e454909d"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "d2763cc57e909702c6d7915225b793a6"
server_key_iv: "46eb5a971e0b99b23179d36064407bd3"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.377559 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "8db20acf55194c63bcbfcf58e454909d"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "d2763cc57e909702c6d7915225b793a6"
server_key_iv: "46eb5a971e0b99b23179d36064407bd3"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.382748 15721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 16:57:13.386346 15752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:13.387730 15721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 16:57:13.387954 15721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "8db20acf55194c63bcbfcf58e454909d"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "d2763cc57e909702c6d7915225b793a6"
server_key_iv: "46eb5a971e0b99b23179d36064407bd3"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.388193 15721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:13.404215 15721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:13.405117 15721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:13.405450 15721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:13.424634 15721 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:34399
I20260812 16:57:13.424659 15853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:34399 every 8 connection(s)
I20260812 16:57:13.426066 15721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:13.428292 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15721
I20260812 16:57:13.428489 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:13.429070 8974 external_mini_cluster.cc:1428] Setting key f85c16ef54babd28ecfdbb780f9db98c
I20260812 16:57:13.430632 15854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:13.442483 15858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:13.431082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.8.195.190:34399)
0812 16:57:13.432041 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:13.432051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:13.432092 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:57:13.433745 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:13.433765 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:13.434198 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:13.434596 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:13.435175 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:13.436518 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:13.437996 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:13.438653 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:13.438856 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:13.440144 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:13.440152 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:13.440166 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:13.440756 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:13.440844 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:13.440867 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:13.441002 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:13.441251 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 16:57:13.441567 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":368,"threads_started":1}
I20260812 16:57:13.442720 15854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: Bootstrap starting.
I20260812 16:57:13.446543 15854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:13.447665 15854 log.cc:824] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:13.450803 15854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: No bootstrap required, opened a new log
I20260812 16:57:13.458631 15854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } }
I20260812 16:57:13.458986 15854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:13.459080 15854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db20acf55194c63bcbfcf58e454909d, State: Initialized, Role: FOLLOWER
I20260812 16:57:13.459631 15854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } }
I20260812 16:57:13.459910 15854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:13.460053 15854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:13.460242 15854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:13.462931 15854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } }
I20260812 16:57:13.463465 15854 leader_election.cc:302] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8db20acf55194c63bcbfcf58e454909d; no voters:
I20260812 16:57:13.464053 15854 leader_election.cc:288] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:13.464215 15861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:13.464978 15861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [term 1 LEADER]: Becoming Leader. State: Replica: 8db20acf55194c63bcbfcf58e454909d, State: Running, Role: LEADER
I20260812 16:57:13.465548 15861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } }
I20260812 16:57:13.466276 15854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:13.468466 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8db20acf55194c63bcbfcf58e454909d. Latest consensus state: current_term: 1 leader_uuid: "8db20acf55194c63bcbfcf58e454909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } } }
I20260812 16:57:13.468889 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:13.471148 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8db20acf55194c63bcbfcf58e454909d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8db20acf55194c63bcbfcf58e454909d" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 34399 } } }
I20260812 16:57:13.471542 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:13.473547 15875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:13.479024 15875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:13.487270 15875 catalog_manager.cc:1347] Generated new cluster ID: 0e54f3d2779847da94fa70b10e55c157
I20260812 16:57:13.487401 15875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:13.504431 15875 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:13.505821 15875 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:13.513767 15875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: Generated new TSK 0
I20260812 16:57:13.514635 15875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:13.530117 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.129:0
--local_ip_for_outbound_sockets=127.8.195.129
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--builtin_ntp_servers=127.8.195.148:34925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 16:57:13.881254 15899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:13.881814 15899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:13.882016 15899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:13.891863 15899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:13.892023 15899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:13.892244 15899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.129
I20260812 16:57:13.906109 15899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.195.129
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.129
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:13.907408 15899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:13.909401 15899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:13.922940 15909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:13.922996 15913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:13.923257 15908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:13.924023 15899 server_base.cc:1034] running on GCE node
I20260812 16:57:13.924676 15899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:13.925841 15899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:13.927062 15899 hybrid_clock.cc:648] HybridClock initialized: now 1786553833927022 us; error 33 us; skew 500 ppm
I20260812 16:57:13.927414 15899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:13.929961 15899 webserver.cc:460] Webserver started at http://127.8.195.129:35363/ using document root <none> and password file <none>
I20260812 16:57:13.930685 15899 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:13.930824 15899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:13.931181 15899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:13.934161 15899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data/instance:
uuid: "416c54f86cf44ecc9f5a3320f1355aad"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "74e3b13c4cb35ae05fdf274c022f8700"
server_key_iv: "4ab6655e26d883459306fc9901c971ce"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.934975 15899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance:
uuid: "416c54f86cf44ecc9f5a3320f1355aad"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "74e3b13c4cb35ae05fdf274c022f8700"
server_key_iv: "4ab6655e26d883459306fc9901c971ce"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.940250 15899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:57:13.943981 15928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:13.945411 15899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:57:13.945592 15899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal
uuid: "416c54f86cf44ecc9f5a3320f1355aad"
format_stamp: "Formatted at 2026-08-12 16:57:13 on dist-test-slave-z8x9"
server_key: "74e3b13c4cb35ae05fdf274c022f8700"
server_key_iv: "4ab6655e26d883459306fc9901c971ce"
server_key_version: "encryptionkey@0"
I20260812 16:57:13.945830 15899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:13.969137 15899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:13.970142 15899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:13.970489 15899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:13.971686 15899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:13.973568 15899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:13.973716 15899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:13.973855 15899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:13.973940 15899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.007020 15899 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.129:44493
I20260812 16:57:14.007120 16160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.129:44493 every 8 connection(s)
I20260812 16:57:14.008349 15899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/data/info.pb
I20260812 16:57:14.010484 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 15899
I20260812 16:57:14.010682 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-0/wal/instance
I20260812 16:57:14.011271 8974 external_mini_cluster.cc:1428] Setting key 5ec99b16669970ca75f50d662805ad2a
I20260812 16:57:14.019936 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.130:0
--local_ip_for_outbound_sockets=127.8.195.130
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--builtin_ntp_servers=127.8.195.148:34925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:57:14.026602 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.012129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.129:42219 (local address 127.8.195.190:34399)
0812 16:57:14.017549 (+ 5420us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:14.017553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:14.017573 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:57:14.018152 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:14.018166 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:14.018240 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:14.018364 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:14.019863 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.020695 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.023290 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.023715 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.023838 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.025611 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:14.025619 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:14.025626 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:14.025652 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:14.025701 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:14.025718 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:14.025812 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:14.026073 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 16:57:14.026181 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5312,"thread_start_us":5228,"threads_started":1}
I20260812 16:57:14.027963 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.011578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34399 (local address 127.8.195.129:42219)
0812 16:57:14.014863 (+ 3285us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:14.014922 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:14.017893 (+ 2971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:14.018606 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:14.018613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:14.018771 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:14.019655 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.019682 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.020885 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.020894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.023116 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.023135 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.023964 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.023973 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.024314 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.025204 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:14.025210 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:14.025271 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:14.025389 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:14.025826 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:14.025840 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:14.025851 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:57:14.026035 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 16:57:14.026476 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2563,"thread_start_us":2341,"threads_started":1}
I20260812 16:57:14.029737 16163 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34399
I20260812 16:57:14.030092 16163 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:14.031065 16163 heartbeater.cc:511] Master 127.8.195.190:34399 requested a full tablet report, sending...
I20260812 16:57:14.033506 15782 ts_manager.cc:194] Registered new tserver with Master: 416c54f86cf44ecc9f5a3320f1355aad (127.8.195.129:44493)
I20260812 16:57:14.035148 15782 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.129:42219
W20260812 16:57:14.367991 16169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:14.368283 16169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:14.368466 16169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:14.375803 16169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:14.375926 16169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:14.376106 16169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.130
I20260812 16:57:14.385342 16169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.195.130
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.130
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:14.386682 16169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:14.388499 16169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:14.402565 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:14.402650 16180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:14.402669 16185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:14.403527 16169 server_base.cc:1034] running on GCE node
I20260812 16:57:14.404138 16169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:14.405143 16169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:14.406414 16169 hybrid_clock.cc:648] HybridClock initialized: now 1786553834406373 us; error 34 us; skew 500 ppm
I20260812 16:57:14.406749 16169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:14.409221 16169 webserver.cc:460] Webserver started at http://127.8.195.130:36949/ using document root <none> and password file <none>
I20260812 16:57:14.409996 16169 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:14.410091 16169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:14.410377 16169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:14.413241 16169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data/instance:
uuid: "0fa5c40f154d48249b0cfc43853c18cc"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "a73858937090fa3345d979e3c827cb91"
server_key_iv: "f77a2088d618e7d7e9022509622716ab"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.414105 16169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance:
uuid: "0fa5c40f154d48249b0cfc43853c18cc"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "a73858937090fa3345d979e3c827cb91"
server_key_iv: "f77a2088d618e7d7e9022509622716ab"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.419116 16169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:14.422569 16197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.424041 16169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:14.424292 16169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal
uuid: "0fa5c40f154d48249b0cfc43853c18cc"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "a73858937090fa3345d979e3c827cb91"
server_key_iv: "f77a2088d618e7d7e9022509622716ab"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.424535 16169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:14.441545 16169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:14.442471 16169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:14.442829 16169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:14.443949 16169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:14.445672 16169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:14.445808 16169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.445951 16169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:14.446064 16169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.483335 16169 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.130:35029
I20260812 16:57:14.483376 16424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.130:35029 every 8 connection(s)
I20260812 16:57:14.484776 16169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/data/info.pb
I20260812 16:57:14.485397 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16169
I20260812 16:57:14.485610 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal/instance
I20260812 16:57:14.486234 8974 external_mini_cluster.cc:1428] Setting key 8d1272b95abad0196ff353c9e20de1bb
I20260812 16:57:14.489934 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.195.131:0
--local_ip_for_outbound_sockets=127.8.195.131
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--builtin_ntp_servers=127.8.195.148:34925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 16:57:14.501348 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.487797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:43299 (local address 127.8.195.190:34399)
0812 16:57:14.488007 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:14.488012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:14.489767 (+ 1755us) server_negotiation.cc:407] Connection header received
0812 16:57:14.492973 (+ 3206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:14.492977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:14.493062 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:14.493215 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:14.494743 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.495472 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.498208 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.498655 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.498754 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.500543 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:14.500547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:14.500550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:14.500569 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:14.500601 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:14.500613 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:14.500677 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:14.500965 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 16:57:14.501047 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 16:57:14.502955 16429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.488426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34399 (local address 127.8.195.130:43299)
0812 16:57:14.489594 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:14.489648 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:14.492612 (+ 2964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:14.493541 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:14.493556 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:14.493681 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:14.494556 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.494586 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.495640 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.495648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.498045 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.498065 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.498909 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.498920 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.499290 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.500143 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:14.500150 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:14.500209 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:14.500331 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:14.500710 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:14.500724 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:14.500735 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 16:57:14.500936 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 16:57:14.501435 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":329,"threads_started":1}
I20260812 16:57:14.504516 16425 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34399
I20260812 16:57:14.504830 16425 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:14.505862 16425 heartbeater.cc:511] Master 127.8.195.190:34399 requested a full tablet report, sending...
I20260812 16:57:14.507488 15782 ts_manager.cc:194] Registered new tserver with Master: 0fa5c40f154d48249b0cfc43853c18cc (127.8.195.130:35029)
I20260812 16:57:14.508375 15782 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.130:43299
W20260812 16:57:14.838649 16431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:14.839000 16431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:14.839167 16431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:14.846560 16431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:14.846710 16431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:14.846890 16431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.195.131
I20260812 16:57:14.856693 16431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:34925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.195.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.195.131
--webserver_port=0
--tserver_master_addrs=127.8.195.190:34399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.195.131
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:14.858307 16431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:14.860227 16431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:14.874181 16441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:14.874148 16442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:14.874097 16446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:14.875149 16431 server_base.cc:1034] running on GCE node
I20260812 16:57:14.875803 16431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:14.876780 16431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:14.878028 16431 hybrid_clock.cc:648] HybridClock initialized: now 1786553834877977 us; error 62 us; skew 500 ppm
I20260812 16:57:14.878374 16431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 16:57:14.880862 16431 webserver.cc:460] Webserver started at http://127.8.195.131:35257/ using document root <none> and password file <none>
I20260812 16:57:14.881646 16431 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:14.881793 16431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:14.882139 16431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:14.885047 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data/instance:
uuid: "26c0cb4c578e4850a469eb8da0ea2d83"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "39368e30f4d7360b0e6a2ded078ae238"
server_key_iv: "f83569e0d39c151a2b0f8fc5c9896074"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.886021 16431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance:
uuid: "26c0cb4c578e4850a469eb8da0ea2d83"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "39368e30f4d7360b0e6a2ded078ae238"
server_key_iv: "f83569e0d39c151a2b0f8fc5c9896074"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.891283 16431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:14.895056 16457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.896502 16431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:57:14.896695 16431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal
uuid: "26c0cb4c578e4850a469eb8da0ea2d83"
format_stamp: "Formatted at 2026-08-12 16:57:14 on dist-test-slave-z8x9"
server_key: "39368e30f4d7360b0e6a2ded078ae238"
server_key_iv: "f83569e0d39c151a2b0f8fc5c9896074"
server_key_version: "encryptionkey@0"
I20260812 16:57:14.896938 16431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:14.912086 16431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:14.912928 16431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:14.913305 16431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:14.914299 16431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 16:57:14.915774 16431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 16:57:14.915895 16431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.916031 16431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 16:57:14.916121 16431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:14.952817 16431 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.131:40427
I20260812 16:57:14.952869 16665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.131:40427 every 8 connection(s)
I20260812 16:57:14.954244 16431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/data/info.pb
I20260812 16:57:14.956816 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16431
I20260812 16:57:14.957018 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal/instance
I20260812 16:57:14.957532 8974 external_mini_cluster.cc:1428] Setting key 131ca41adefd1c21244007c72da0c812
I20260812 16:57:14.967039 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.956937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.131:41233 (local address 127.8.195.190:34399)
0812 16:57:14.957189 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:14.957196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:14.958723 (+ 1527us) server_negotiation.cc:407] Connection header received
0812 16:57:14.960500 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:14.960504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:14.960568 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:14.960651 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:14.961739 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.962630 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.964160 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.964564 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.964665 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.966231 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:14.966236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:14.966240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:14.966259 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:14.966304 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:14.966317 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:14.966394 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:14.966621 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 16:57:14.966708 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 16:57:14.968035 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.957481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:34399 (local address 127.8.195.131:41233)
0812 16:57:14.958555 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:14.958607 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:14.960279 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:14.960874 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:14.960881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:14.960966 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:57:14.961594 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.961611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.962778 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.962781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.964024 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:14.964037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.964751 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.964756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:14.964982 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.965826 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 16:57:14.965829 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:57:14.965870 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 16:57:14.966025 (+ 155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:57:14.966408 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:57:14.966416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:57:14.966422 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 16:57:14.966562 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 16:57:14.966902 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":398,"threads_started":1}
I20260812 16:57:14.969508 16666 heartbeater.cc:347] Connected to a master server at 127.8.195.190:34399
I20260812 16:57:14.969856 16666 heartbeater.cc:464] Registering TS with master...
I20260812 16:57:14.970651 16666 heartbeater.cc:511] Master 127.8.195.190:34399 requested a full tablet report, sending...
I20260812 16:57:14.972174 15782 ts_manager.cc:194] Registered new tserver with Master: 26c0cb4c578e4850a469eb8da0ea2d83 (127.8.195.131:40427)
I20260812 16:57:14.972853 8974 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 16:57:14.973270 15782 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.195.131:41233
I20260812 16:57:14.983302 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:14.977571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.8.195.190:34399)
0812 16:57:14.977835 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:14.977845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:14.978183 (+ 338us) server_negotiation.cc:407] Connection header received
0812 16:57:14.978332 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:14.978336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:14.978407 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:14.978487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:14.978940 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.979622 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.980972 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:14.981461 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:14.981560 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:14.982593 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:14.982599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:14.982602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:14.982619 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:14.982651 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:14.982662 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:14.982739 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:14.982859 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 16:57:14.982936 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:57:14.987435 15782 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 16:57:14.994549 15782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 16:57:15.032639 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.007539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.130:35029 (local address 127.0.0.1:44012)
0812 16:57:15.020502 (+ 12963us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.020564 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:15.020774 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:15.024647 (+ 3873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:15.024653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:15.024680 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:15.025276 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.025297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027486 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.027495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.030237 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.030249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.031593 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.031598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.031840 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.031892 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 16:57:15.032009 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 16:57:15.032211 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12637,"mutex_wait_us":198,"thread_start_us":16407,"threads_started":1}
I20260812 16:57:15.032608 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.008105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.8.195.130:35029)
0812 16:57:15.024061 (+ 15956us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.024069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.024098 (+ 29us) server_negotiation.cc:407] Connection header received
0812 16:57:15.024211 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.024219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.024442 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.024647 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:15.025769 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027286 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.030457 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.031361 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.031778 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.032001 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.032092 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.032254 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15801,"thread_start_us":15651,"threads_started":1}
I20260812 16:57:15.033215 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.022690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40427 (local address 127.0.0.1:47670)
0812 16:57:15.024128 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.024148 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:15.024266 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:15.024927 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:15.024932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:15.024961 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:15.025520 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.025535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027549 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.027555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.030190 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.030207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.032369 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.032376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.032546 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.032580 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 16:57:15.032664 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 16:57:15.032797 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1313,"mutex_wait_us":8,"thread_start_us":328,"threads_started":1}
I20260812 16:57:15.033847 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.021454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47670 (local address 127.8.195.131:40427)
0812 16:57:15.023729 (+ 2275us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.023740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.024402 (+ 662us) server_negotiation.cc:407] Connection header received
0812 16:57:15.024509 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.024516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.024717 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.024896 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:15.025738 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027396 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.030422 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.031803 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.031962 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.032239 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.033334 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.033518 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2101,"thread_start_us":1938,"threads_started":1}
I20260812 16:57:15.034914 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.007870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.8.195.129:44493)
0812 16:57:15.022610 (+ 14740us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.022622 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.023501 (+ 879us) server_negotiation.cc:407] Connection header received
0812 16:57:15.023833 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.023842 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.024123 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.024297 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:15.025530 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027217 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.029984 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.031341 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.031484 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.031782 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.033532 (+ 1750us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.033703 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14513,"thread_start_us":14409,"threads_started":1}
I20260812 16:57:15.035595 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.007561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49836)
0812 16:57:15.023375 (+ 15814us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.023398 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:15.023622 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:15.024552 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:15.024557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:15.024613 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:15.025309 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.025320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.027368 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.027373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.029791 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.029805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.031567 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.031581 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.033321 (+ 1740us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.033358 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 16:57:15.035117 (+ 1759us) client_negotiation.cc:240] Negotiation successful
0812 16:57:15.035267 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15637,"mutex_wait_us":230,"thread_start_us":12457,"threads_started":1}
I20260812 16:57:15.036990 16551 tablet_service.cc:1467] Processing CreateTablet for tablet 0a19a55ae249450b9e0156b476269d43 (DEFAULT_TABLE table=test-table [id=1db59332272f404eb3275041846f563f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:15.036986 16289 tablet_service.cc:1467] Processing CreateTablet for tablet 0a19a55ae249450b9e0156b476269d43 (DEFAULT_TABLE table=test-table [id=1db59332272f404eb3275041846f563f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:15.038473 16551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a19a55ae249450b9e0156b476269d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:15.038451 16163 heartbeater.cc:503] Master 127.8.195.190:34399 was elected leader, sending a full tablet report...
I20260812 16:57:15.038493 16289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a19a55ae249450b9e0156b476269d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:15.040743 16021 tablet_service.cc:1467] Processing CreateTablet for tablet 0a19a55ae249450b9e0156b476269d43 (DEFAULT_TABLE table=test-table [id=1db59332272f404eb3275041846f563f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 16:57:15.043417 16021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a19a55ae249450b9e0156b476269d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:15.049875 16700 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Bootstrap starting.
I20260812 16:57:15.053092 16702 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Bootstrap starting.
I20260812 16:57:15.053821 16700 tablet_bootstrap.cc:654] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:15.055159 16700 log.cc:824] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:15.056700 16702 tablet_bootstrap.cc:654] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:15.058030 16702 log.cc:824] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:15.058576 16700 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: No bootstrap required, opened a new log
I20260812 16:57:15.058883 16705 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Bootstrap starting.
I20260812 16:57:15.059015 16700 ts_tablet_manager.cc:1397] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 16:57:15.062328 16705 tablet_bootstrap.cc:654] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:15.063572 16705 log.cc:824] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:15.065063 16702 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: No bootstrap required, opened a new log
I20260812 16:57:15.065434 16702 ts_tablet_manager.cc:1397] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
I20260812 16:57:15.066123 16705 tablet_bootstrap.cc:492] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: No bootstrap required, opened a new log
I20260812 16:57:15.066475 16705 ts_tablet_manager.cc:1397] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260812 16:57:15.070477 16700 raft_consensus.cc:348] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.070988 16700 raft_consensus.cc:374] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:15.071147 16700 raft_consensus.cc:729] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa5c40f154d48249b0cfc43853c18cc, State: Initialized, Role: FOLLOWER
I20260812 16:57:15.072099 16700 consensus_queue.cc:260] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.072593 16710 raft_consensus.cc:482] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 16:57:15.072978 16710 raft_consensus.cc:504] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.075243 16705 raft_consensus.cc:348] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.075773 16705 raft_consensus.cc:374] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:15.075932 16705 raft_consensus.cc:729] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416c54f86cf44ecc9f5a3320f1355aad, State: Initialized, Role: FOLLOWER
I20260812 16:57:15.076769 16705 consensus_queue.cc:260] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.078048 16702 raft_consensus.cc:348] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.078660 16702 raft_consensus.cc:374] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:15.078845 16702 raft_consensus.cc:729] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c0cb4c578e4850a469eb8da0ea2d83, State: Initialized, Role: FOLLOWER
I20260812 16:57:15.079840 16425 heartbeater.cc:503] Master 127.8.195.190:34399 was elected leader, sending a full tablet report...
I20260812 16:57:15.079703 16702 consensus_queue.cc:260] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.080559 16700 ts_tablet_manager.cc:1428] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Time spent starting tablet: real 0.021s user 0.012s sys 0.007s
I20260812 16:57:15.081682 16666 heartbeater.cc:503] Master 127.8.195.190:34399 was elected leader, sending a full tablet report...
I20260812 16:57:15.082803 16705 ts_tablet_manager.cc:1428] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260812 16:57:15.083796 16702 ts_tablet_manager.cc:1428] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260812 16:57:15.085114 16710 leader_election.cc:288] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 416c54f86cf44ecc9f5a3320f1355aad (127.8.195.129:44493), 26c0cb4c578e4850a469eb8da0ea2d83 (127.8.195.131:40427)
I20260812 16:57:15.093436 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.078317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:46089 (local address 127.8.195.129:44493)
0812 16:57:15.078556 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.078564 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.086115 (+ 7551us) server_negotiation.cc:407] Connection header received
0812 16:57:15.086411 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.086417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.086508 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.086577 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:15.087459 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.088495 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.091140 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.092341 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.092464 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.092512 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.093029 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.093157 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:57:15.093566 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.077669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.8.195.130:46089)
0812 16:57:15.085999 (+ 8330us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.086028 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:15.086234 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:15.086776 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:15.086780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:15.086817 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:15.087264 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.087279 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.088679 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.088683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.090925 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.090936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.092539 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.092545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.092727 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.092769 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 16:57:15.092976 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 16:57:15.093149 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8166,"mutex_wait_us":1,"thread_start_us":7830,"threads_started":1}
I20260812 16:57:15.094762 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a19a55ae249450b9e0156b476269d43" candidate_uuid: "0fa5c40f154d48249b0cfc43853c18cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "416c54f86cf44ecc9f5a3320f1355aad" is_pre_election: true
I20260812 16:57:15.095400 16073 raft_consensus.cc:2454] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa5c40f154d48249b0cfc43853c18cc in term 0.
I20260812 16:57:15.096192 16201 leader_election.cc:302] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa5c40f154d48249b0cfc43853c18cc, 416c54f86cf44ecc9f5a3320f1355aad; no voters:
I20260812 16:57:15.096556 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.086293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.195.130:39607 (local address 127.8.195.131:40427)
0812 16:57:15.086474 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.086480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.089774 (+ 3294us) server_negotiation.cc:407] Connection header received
0812 16:57:15.090047 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.090052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.090128 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.090250 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 16:57:15.090866 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.091935 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.094437 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.095655 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.095855 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.095942 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.096204 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.096333 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:57:15.096585 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.086618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.131:40427 (local address 127.8.195.130:39607)
0812 16:57:15.089667 (+ 3049us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.089688 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:57:15.089885 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:57:15.090245 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:57:15.090250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:57:15.090279 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 16:57:15.090696 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.090708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.092085 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.092090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.094235 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:57:15.094250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.095867 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.095877 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:57:15.096048 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.096072 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 16:57:15.096144 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 16:57:15.096235 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2907,"mutex_wait_us":28,"thread_start_us":2753,"threads_started":1}
I20260812 16:57:15.096676 16710 raft_consensus.cc:2781] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 16:57:15.096835 16710 raft_consensus.cc:482] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 16:57:15.096956 16710 raft_consensus.cc:3037] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:15.097786 16593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a19a55ae249450b9e0156b476269d43" candidate_uuid: "0fa5c40f154d48249b0cfc43853c18cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" is_pre_election: true
I20260812 16:57:15.098359 16593 raft_consensus.cc:2454] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa5c40f154d48249b0cfc43853c18cc in term 0.
I20260812 16:57:15.100033 16710 raft_consensus.cc:504] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.100697 16710 leader_election.cc:288] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [CANDIDATE]: Term 1 election: Requested vote from peers 416c54f86cf44ecc9f5a3320f1355aad (127.8.195.129:44493), 26c0cb4c578e4850a469eb8da0ea2d83 (127.8.195.131:40427)
I20260812 16:57:15.101159 16073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a19a55ae249450b9e0156b476269d43" candidate_uuid: "0fa5c40f154d48249b0cfc43853c18cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "416c54f86cf44ecc9f5a3320f1355aad"
I20260812 16:57:15.101315 16593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a19a55ae249450b9e0156b476269d43" candidate_uuid: "0fa5c40f154d48249b0cfc43853c18cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26c0cb4c578e4850a469eb8da0ea2d83"
I20260812 16:57:15.101426 16073 raft_consensus.cc:3037] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:15.101547 16593 raft_consensus.cc:3037] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:15.104813 16073 raft_consensus.cc:2454] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa5c40f154d48249b0cfc43853c18cc in term 1.
I20260812 16:57:15.104813 16593 raft_consensus.cc:2454] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fa5c40f154d48249b0cfc43853c18cc in term 1.
I20260812 16:57:15.105254 16201 leader_election.cc:302] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa5c40f154d48249b0cfc43853c18cc, 416c54f86cf44ecc9f5a3320f1355aad; no voters:
I20260812 16:57:15.105590 16710 raft_consensus.cc:2781] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:15.105768 16710 raft_consensus.cc:686] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 1 LEADER]: Becoming Leader. State: Replica: 0fa5c40f154d48249b0cfc43853c18cc, State: Running, Role: LEADER
I20260812 16:57:15.106273 16710 consensus_queue.cc:237] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } }
I20260812 16:57:15.111394 15782 catalog_manager.cc:5495] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fa5c40f154d48249b0cfc43853c18cc (127.8.195.130). New cstate: current_term: 1 leader_uuid: "0fa5c40f154d48249b0cfc43853c18cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fa5c40f154d48249b0cfc43853c18cc" member_type: VOTER last_known_addr { host: "127.8.195.130" port: 35029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 } health_report { overall_health: UNKNOWN } } }
I20260812 16:57:15.148689 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.143503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.8.195.130:35029)
0812 16:57:15.143783 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.143787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.143806 (+ 19us) server_negotiation.cc:407] Connection header received
0812 16:57:15.143992 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.143996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.144100 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.144272 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:57:15.144570 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.145354 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.146681 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.147087 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.147182 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.147623 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:57:15.148130 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:57:15.148265 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.148310 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.148372 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 16:57:15.157711 16073 raft_consensus.cc:1261] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 1 FOLLOWER]: Refusing update from remote peer 0fa5c40f154d48249b0cfc43853c18cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:15.157717 16593 raft_consensus.cc:1261] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 1 FOLLOWER]: Refusing update from remote peer 0fa5c40f154d48249b0cfc43853c18cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:57:15.158805 16711 consensus_queue.cc:1035] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "416c54f86cf44ecc9f5a3320f1355aad" member_type: VOTER last_known_addr { host: "127.8.195.129" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:15.159214 16710 consensus_queue.cc:1035] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c0cb4c578e4850a469eb8da0ea2d83" member_type: VOTER last_known_addr { host: "127.8.195.131" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:57:15.171857 16721 mvcc.cc:205] Tried to move back new op lower bound from 7317724508794863616 to 7317724508599578624. Current Snapshot: MvccSnapshot[applied={T|T < 7317724508794863616}]
I20260812 16:57:15.186866 15777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:15.187199 15777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 16:57:15.191267 15777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8db20acf55194c63bcbfcf58e454909d: Sending DeleteTablet for 3 replicas of tablet 0a19a55ae249450b9e0156b476269d43
I20260812 16:57:15.192577 16289 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a19a55ae249450b9e0156b476269d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:15 UTC) from {username='slave'} at 127.0.0.1:44012
I20260812 16:57:15.192716 16551 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a19a55ae249450b9e0156b476269d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:15 UTC) from {username='slave'} at 127.0.0.1:47670
I20260812 16:57:15.193434 16021 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a19a55ae249450b9e0156b476269d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 16:57:15 UTC) from {username='slave'} at 127.0.0.1:49836
I20260812 16:57:15.194365 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15899
I20260812 16:57:15.220248 16744 tablet_replica.cc:330] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: stopping tablet replica
I20260812 16:57:15.222597 16744 raft_consensus.cc:2229] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:15.223518 16741 tablet_replica.cc:330] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: stopping tablet replica
I20260812 16:57:15.223771 16742 tablet_replica.cc:330] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: stopping tablet replica
I20260812 16:57:15.224274 16744 raft_consensus.cc:2258] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:15.224270 16742 raft_consensus.cc:2229] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 1 LEADER]: Raft consensus shutting down.
I20260812 16:57:15.224809 16741 raft_consensus.cc:2229] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 16:57:15.225001 16742 raft_consensus.cc:2258] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:15.226081 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.195328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.8.195.129:44493)
0812 16:57:15.195559 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.195568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.216490 (+ 20922us) server_negotiation.cc:407] Connection header received
0812 16:57:15.216729 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.216736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.216903 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.217036 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:15.217784 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.219369 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.221686 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.222383 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.222496 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.224214 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:15.224227 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:15.224247 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:15.225050 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:15.225115 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:15.225140 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:15.225278 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.225544 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.225674 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 16:57:15.227371 16742 ts_tablet_manager.cc:1905] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:15.227528 16151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:15.227873 16741 raft_consensus.cc:2258] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 16:57:15.230298 16741 ts_tablet_manager.cc:1905] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:15.235220 16742 ts_tablet_manager.cc:1918] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:15.235423 16742 log.cc:1196] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-1/wal/wals/0a19a55ae249450b9e0156b476269d43
I20260812 16:57:15.235991 16742 ts_tablet_manager.cc:1939] T 0a19a55ae249450b9e0156b476269d43 P 0fa5c40f154d48249b0cfc43853c18cc: Deleting consensus metadata
I20260812 16:57:15.237987 15753 catalog_manager.cc:4841] TS 0fa5c40f154d48249b0cfc43853c18cc (127.8.195.130:35029): tablet 0a19a55ae249450b9e0156b476269d43 (table test-table [id=1db59332272f404eb3275041846f563f]) successfully deleted
I20260812 16:57:15.238544 16741 ts_tablet_manager.cc:1918] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 16:57:15.238759 16741 log.cc:1196] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Deleting WAL directory at /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786553491558846-8974-0/minicluster-data/ts-2/wal/wals/0a19a55ae249450b9e0156b476269d43
I20260812 16:57:15.239363 16741 ts_tablet_manager.cc:1939] T 0a19a55ae249450b9e0156b476269d43 P 26c0cb4c578e4850a469eb8da0ea2d83: Deleting consensus metadata
I20260812 16:57:15.241381 15757 catalog_manager.cc:4841] TS 26c0cb4c578e4850a469eb8da0ea2d83 (127.8.195.131:40427): tablet 0a19a55ae249450b9e0156b476269d43 (table test-table [id=1db59332272f404eb3275041846f563f]) successfully deleted
I20260812 16:57:15.230485 16744 ts_tablet_manager.cc:1905] T 0a19a55ae249450b9e0156b476269d43 P 416c54f86cf44ecc9f5a3320f1355aad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 16:57:15.328619 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15899
I20260812 16:57:15.358870 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16169
W20260812 16:57:15.359160 15757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.195.129:44493 (error 108)
I20260812 16:57:15.361759 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.361162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49854)
0812 16:57:15.361421 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.361567 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:57:15.362056 15757 catalog_manager.cc:4584] TS 416c54f86cf44ecc9f5a3320f1355aad (127.8.195.129:44493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a19a55ae249450b9e0156b476269d43: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
I20260812 16:57:15.367077 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.359845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.8.195.130:35029)
0812 16:57:15.360025 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.360033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.360074 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:57:15.360342 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.360348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.360487 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.360588 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:15.361278 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.362497 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.363756 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.364195 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.364346 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.365562 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:15.365571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:15.365585 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:15.366220 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:15.366273 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:15.366295 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:15.366431 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.366564 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.366681 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:57:15.367920 16420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:15.415946 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.415398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49860)
0812 16:57:15.415674 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.415778 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260812 16:57:15.416999 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.416676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49862)
0812 16:57:15.416803 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.416876 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 16:57:15.477667 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.477086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49876)
0812 16:57:15.477369 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.477468 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260812 16:57:15.478741 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.478422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49880)
0812 16:57:15.478556 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.478630 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 16:57:15.494345 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16169
I20260812 16:57:15.521085 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16431
I20260812 16:57:15.527930 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.521956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.8.195.131:40427)
0812 16:57:15.522147 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:15.522152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:15.522172 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:57:15.522272 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:15.522276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:15.522362 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:15.522449 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:15.522838 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.523690 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.525001 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:15.525500 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:15.525600 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:15.526612 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:15.526621 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:15.526636 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:15.527211 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:15.527257 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:15.527276 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:15.527366 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:15.527483 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 16:57:15.527607 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:57:15.528587 16661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:15.559576 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.559070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49884)
0812 16:57:15.559320 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.559411 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 16:57:15.560684 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:15.560256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.129:44493 (local address 127.0.0.1:49900)
0812 16:57:15.560469 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 16:57:15.560576 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.129:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 16:57:15.619306 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16431
I20260812 16:57:15.643281 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 15721
I20260812 16:57:15.644471 15845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:15.767685 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 15721
2026-08-12T16:57:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2901 ms)
[----------] 6 tests from EncryptionPolicyTest (23909 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:15 dist-test-slave-z8x9 krb5kdc[16760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:15 dist-test-slave-z8x9 krb5kdc[16760](info): set up 2 sockets
Aug 12 16:57:15 dist-test-slave-z8x9 krb5kdc[16760](info): commencing operation
krb5kdc: starting...
W20260812 16:57:17.226687 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.400s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:17 dist-test-slave-z8x9 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:17.567087 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:17.921028 16780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:17.921448 16780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:17.921625 16780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:17.929564 16780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:17.929688 16780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:17.929792 16780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:17.929901 16780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:17.929986 16780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:17.940099 16780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39361
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:17.941493 16780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:17.942847 16780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:17.955219 16796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:17.955302 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:17.955271 16785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:17.956547 16780 server_base.cc:1034] running on GCE node
I20260812 16:57:17.957533 16780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:17.959045 16780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:17.960270 16780 hybrid_clock.cc:648] HybridClock initialized: now 1786553837960206 us; error 49 us; skew 500 ppm
I20260812 16:57:17.960682 16780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:17 dist-test-slave-z8x9 krb5kdc[16760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:17.969533 16780 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:17.971246 16780 webserver.cc:460] Webserver started at http://127.0.0.1:36927/ using document root <none> and password file <none>
I20260812 16:57:17.972013 16780 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:17.972169 16780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:17.972496 16780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:17.975760 16780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "30b01e2115594f1db17d02752cc2cec9"
format_stamp: "Formatted at 2026-08-12 16:57:17 on dist-test-slave-z8x9"
server_key: "d40bc3310f13e9dd03bd045fc4e13d24"
server_key_iv: "93e059fc474ff5ed7239de7abb3965c0"
server_key_version: "encryptionkey@0"
I20260812 16:57:17.976580 16780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "30b01e2115594f1db17d02752cc2cec9"
format_stamp: "Formatted at 2026-08-12 16:57:17 on dist-test-slave-z8x9"
server_key: "d40bc3310f13e9dd03bd045fc4e13d24"
server_key_iv: "93e059fc474ff5ed7239de7abb3965c0"
server_key_version: "encryptionkey@0"
I20260812 16:57:17.982276 16780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 16:57:17.986040 16810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:17.987386 16780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:17.987591 16780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "30b01e2115594f1db17d02752cc2cec9"
format_stamp: "Formatted at 2026-08-12 16:57:17 on dist-test-slave-z8x9"
server_key: "d40bc3310f13e9dd03bd045fc4e13d24"
server_key_iv: "93e059fc474ff5ed7239de7abb3965c0"
server_key_version: "encryptionkey@0"
I20260812 16:57:17.987818 16780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:18.011293 16780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:18.016106 16780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:18.016426 16780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:18.039739 16780 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39361
I20260812 16:57:18.039834 16909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39361 every 8 connection(s)
I20260812 16:57:18.041157 16780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:18.044987 16915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:18.051056 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 16780
I20260812 16:57:18.051297 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:18.052038 8974 external_mini_cluster.cc:1428] Setting key fe21e91b2539c3f729972e75eecb170e
I20260812 16:57:18.056603 16915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9: Bootstrap starting.
I20260812 16:57:18.060781 16915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:18.062155 16915 log.cc:824] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:18.065923 16915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9: No bootstrap required, opened a new log
Aug 12 16:57:18 dist-test-slave-z8x9 krb5kdc[16760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:57:18.076016 16915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } }
I20260812 16:57:18.076524 16915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:18.076644 16915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b01e2115594f1db17d02752cc2cec9, State: Initialized, Role: FOLLOWER
I20260812 16:57:18.077502 16915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } }
I20260812 16:57:18.077843 16915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:18.078061 16915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:18.078298 16915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:18.079900 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:18.054317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35505 (local address 127.0.0.1:39361)
0812 16:57:18.055138 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:18.055164 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:18.055217 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:57:18.057668 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:18.057697 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:18.058197 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:18.058701 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:18.060178 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:18.062333 (+ 2155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:18.064113 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:18.065276 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:18.065493 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:18.069957 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:18.069974 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:18.069998 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:18.070073 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:18.074676 (+ 4603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:18.075665 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:18.075674 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:18.075683 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:18.075907 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:18.076380 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:18.076385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:18.076387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:18.077209 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:18.077552 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:18.078018 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 16:57:18.078414 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":360,"threads_started":1}
I20260812 16:57:18.081552 16915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } }
I20260812 16:57:18.082100 16915 leader_election.cc:302] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30b01e2115594f1db17d02752cc2cec9; no voters:
I20260812 16:57:18.082765 16915 leader_election.cc:288] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:18.082902 16922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:18.084674 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:18.084808 16922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [term 1 LEADER]: Becoming Leader. State: Replica: 30b01e2115594f1db17d02752cc2cec9, State: Running, Role: LEADER
I20260812 16:57:18.085242 16915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:18.085465 16922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } }
I20260812 16:57:18.093494 16924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30b01e2115594f1db17d02752cc2cec9. Latest consensus state: current_term: 1 leader_uuid: "30b01e2115594f1db17d02752cc2cec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } } }
I20260812 16:57:18.093838 16924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:18.093308 16923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30b01e2115594f1db17d02752cc2cec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b01e2115594f1db17d02752cc2cec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39361 } } }
I20260812 16:57:18.094756 16923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:18.097757 16942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:18.103559 16942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:18.114943 16942 catalog_manager.cc:1347] Generated new cluster ID: ab2263aa971d451abe4f92e2e6596ab0
I20260812 16:57:18.115065 16942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:18.117029 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:18.093683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58565 (local address 127.0.0.1:39361)
0812 16:57:18.093885 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:18.093893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:18.098052 (+ 4159us) server_negotiation.cc:407] Connection header received
0812 16:57:18.098396 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:18.098405 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:18.098512 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:18.098609 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:18.100814 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:18.102278 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:18.104216 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:18.104929 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:18.105058 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:18.108768 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:18.108776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:18.108782 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:18.108834 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:18.113253 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:18.114492 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:18.114498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:18.114501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:18.114682 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:18.115316 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:18.115326 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:18.115328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:18.115727 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:18.116019 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:18.116412 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 16:57:18.116628 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:57:18.148344 16942 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:18.150122 16942 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:18.159746 16942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30b01e2115594f1db17d02752cc2cec9: Generated new TSK 0
I20260812 16:57:18.160954 16942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:18.170364 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 16780
I20260812 16:57:18.171638 16901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:18.264528 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 16780
W20260812 16:57:18.291812 16756 connection.cc:535] client connection to 127.0.0.1:39361 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39361): Connection reset by peer (error 104)
2026-08-12T16:57:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:18 dist-test-slave-z8x9 krb5kdc[16972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:18 dist-test-slave-z8x9 krb5kdc[16972](info): set up 2 sockets
Aug 12 16:57:18 dist-test-slave-z8x9 krb5kdc[16972](info): commencing operation
krb5kdc: starting...
W20260812 16:57:19.688694 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:19 dist-test-slave-z8x9 krb5kdc[16972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:20.032226 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:20.386602 17000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:20.387002 17000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:20.387176 17000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:20.394470 17000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:20.394641 17000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:20.394770 17000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:20.394882 17000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:20.394973 17000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:20.404901 17000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:20.406322 17000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:20.407656 17000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:20.420001 17010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:20.420004 17011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:20.420001 17018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:20.420936 17000 server_base.cc:1034] running on GCE node
I20260812 16:57:20.421864 17000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:20.423321 17000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:20.424573 17000 hybrid_clock.cc:648] HybridClock initialized: now 1786553840424523 us; error 44 us; skew 500 ppm
I20260812 16:57:20.424965 17000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:20 dist-test-slave-z8x9 krb5kdc[16972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:20.428411 17000 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:20.430049 17000 webserver.cc:460] Webserver started at http://127.0.0.1:44937/ using document root <none> and password file <none>
I20260812 16:57:20.430769 17000 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:20.430922 17000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:20.431238 17000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:20.434432 17000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "3046e9b6602343ea9fc8a1e47044925e"
format_stamp: "Formatted at 2026-08-12 16:57:20 on dist-test-slave-z8x9"
server_key: "148f452d57a7ce2297039562584dfa93"
server_key_iv: "b070412944150e88d04a71c8fbe53a58"
server_key_version: "encryptionkey@0"
I20260812 16:57:20.435209 17000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "3046e9b6602343ea9fc8a1e47044925e"
format_stamp: "Formatted at 2026-08-12 16:57:20 on dist-test-slave-z8x9"
server_key: "148f452d57a7ce2297039562584dfa93"
server_key_iv: "b070412944150e88d04a71c8fbe53a58"
server_key_version: "encryptionkey@0"
I20260812 16:57:20.440498 17000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:57:20.444075 17034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:20.445531 17000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:57:20.445724 17000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "3046e9b6602343ea9fc8a1e47044925e"
format_stamp: "Formatted at 2026-08-12 16:57:20 on dist-test-slave-z8x9"
server_key: "148f452d57a7ce2297039562584dfa93"
server_key_iv: "b070412944150e88d04a71c8fbe53a58"
server_key_version: "encryptionkey@0"
I20260812 16:57:20.445928 17000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:20.463269 17000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:20.467986 17000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:20.468292 17000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:20.488224 17000 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39853
I20260812 16:57:20.488341 17143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39853 every 8 connection(s)
I20260812 16:57:20.489746 17000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:20.493721 17144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:20.496448 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17000
I20260812 16:57:20.496654 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:20.497308 8974 external_mini_cluster.cc:1428] Setting key 3ea56f077d8de408bd29bf487267d0b9
I20260812 16:57:20.507652 17144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e: Bootstrap starting.
I20260812 16:57:20.511716 17144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:20.511909 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:20.499288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49983 (local address 127.0.0.1:39853)
0812 16:57:20.500105 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:20.500120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:20.500166 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:57:20.501965 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:20.501988 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:20.502378 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:20.502699 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:20.503420 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:20.504852 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:20.506953 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:20.507658 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:20.507852 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:20.509338 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:20.509346 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:20.509360 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:20.509954 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:20.510111 (+ 157us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:20.510139 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:20.510276 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:20.510616 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 16:57:20.510832 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":369,"threads_started":1}
I20260812 16:57:20.513079 17144 log.cc:824] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:20.516522 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:20.516885 17144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e: No bootstrap required, opened a new log
I20260812 16:57:20.525935 17144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260812 16:57:20.526513 17144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:20.526674 17144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3046e9b6602343ea9fc8a1e47044925e, State: Initialized, Role: FOLLOWER
I20260812 16:57:20.527422 17144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260812 16:57:20.527716 17144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:20.527895 17144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:20.528111 17144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:20.528699 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:20.521426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44023 (local address 127.0.0.1:39853)
0812 16:57:20.521641 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:20.521647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:20.522093 (+ 446us) server_negotiation.cc:407] Connection header received
0812 16:57:20.522259 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:20.522267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:20.522369 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:20.522485 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:20.522971 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:20.524182 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:20.525642 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:20.526278 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:20.526435 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:20.527791 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:20.527799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:20.527804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:20.527829 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:20.527890 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:20.527916 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:20.528044 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:20.528269 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 16:57:20.528341 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:57:20.531200 17144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260812 16:57:20.531746 17144 leader_election.cc:302] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3046e9b6602343ea9fc8a1e47044925e; no voters:
I20260812 16:57:20.532359 17144 leader_election.cc:288] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:20.532523 17170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:20.533314 17170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [term 1 LEADER]: Becoming Leader. State: Replica: 3046e9b6602343ea9fc8a1e47044925e, State: Running, Role: LEADER
I20260812 16:57:20.533947 17170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260812 16:57:20.534688 17144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:20.538977 17173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3046e9b6602343ea9fc8a1e47044925e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } } }
I20260812 16:57:20.539461 17173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:20.538966 17176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3046e9b6602343ea9fc8a1e47044925e. Latest consensus state: current_term: 1 leader_uuid: "3046e9b6602343ea9fc8a1e47044925e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3046e9b6602343ea9fc8a1e47044925e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } } }
I20260812 16:57:20.539861 17176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:20.540323 17183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:20.551080 17183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:20.562064 17183 catalog_manager.cc:1347] Generated new cluster ID: d153bfafb26d4b98b8ce8454f7fefea4
I20260812 16:57:20.562218 17183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:20.575357 17183 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:20.576745 17183 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:20.584589 17183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3046e9b6602343ea9fc8a1e47044925e: Generated new TSK 0
I20260812 16:57:20.585587 17183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:20.592994 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17000
W20260812 16:57:20.594949 17138 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49983
W20260812 16:57:20.595917 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17000: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:20.596122 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17000
W20260812 16:57:20.628103 16969 connection.cc:535] client connection to 127.0.0.1:39853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39853): Connection reset by peer (error 104)
2026-08-12T16:57:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:20 dist-test-slave-z8x9 krb5kdc[17213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:20 dist-test-slave-z8x9 krb5kdc[17213](info): set up 2 sockets
Aug 12 16:57:20 dist-test-slave-z8x9 krb5kdc[17213](info): commencing operation
krb5kdc: starting...
W20260812 16:57:22.034505 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.387s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:22 dist-test-slave-z8x9 krb5kdc[17213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:22.381052 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:22.733621 17236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:22.733929 17236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:22.734102 17236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:22.741348 17236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:22.741498 17236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:22.741587 17236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:22.741698 17236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:22.741794 17236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:22.751932 17236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:22.753319 17236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:22.754650 17236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:22.766778 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:22.766832 17255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:22.766863 17250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:22.767584 17236 server_base.cc:1034] running on GCE node
I20260812 16:57:22.768513 17236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:22.770011 17236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:22.771253 17236 hybrid_clock.cc:648] HybridClock initialized: now 1786553842771200 us; error 53 us; skew 500 ppm
I20260812 16:57:22.771625 17236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:22 dist-test-slave-z8x9 krb5kdc[17213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:22.774912 17236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:22.776451 17236 webserver.cc:460] Webserver started at http://127.0.0.1:41999/ using document root <none> and password file <none>
I20260812 16:57:22.777287 17236 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:22.777448 17236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:22.777773 17236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:22.780943 17236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6"
format_stamp: "Formatted at 2026-08-12 16:57:22 on dist-test-slave-z8x9"
server_key: "f765bc6507ce7c91b8e1ec140801bcd5"
server_key_iv: "837bb2a447d320775fa12893d6ad56da"
server_key_version: "encryptionkey@0"
I20260812 16:57:22.781831 17236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6"
format_stamp: "Formatted at 2026-08-12 16:57:22 on dist-test-slave-z8x9"
server_key: "f765bc6507ce7c91b8e1ec140801bcd5"
server_key_iv: "837bb2a447d320775fa12893d6ad56da"
server_key_version: "encryptionkey@0"
I20260812 16:57:22.787151 17236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:22.790774 17268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:22.792109 17236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 16:57:22.792303 17236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6"
format_stamp: "Formatted at 2026-08-12 16:57:22 on dist-test-slave-z8x9"
server_key: "f765bc6507ce7c91b8e1ec140801bcd5"
server_key_iv: "837bb2a447d320775fa12893d6ad56da"
server_key_version: "encryptionkey@0"
I20260812 16:57:22.792518 17236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:22.798514 17236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:22.802959 17236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:22.803278 17236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:22.823419 17236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46143
I20260812 16:57:22.823477 17381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46143 every 8 connection(s)
I20260812 16:57:22.824962 17236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:22.828805 17382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:22.829748 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17236
I20260812 16:57:22.829917 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:22.830394 8974 external_mini_cluster.cc:1428] Setting key dd4f964f2de456bb92cbc63e222b96ff
I20260812 16:57:22.839581 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:22.832371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41455 (local address 127.0.0.1:46143)
0812 16:57:22.833230 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:22.833246 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:22.833299 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:57:22.835594 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:22.835624 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:22.836101 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:22.836574 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:22.837615 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:22.837623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:22.837640 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:22.837656 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:22.837724 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:22.837756 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:22.837972 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:22.838099 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 16:57:22.838305 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":340,"threads_started":1}
I20260812 16:57:22.843734 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:22.844144 17382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6: Bootstrap starting.
I20260812 16:57:22.848263 17382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:22.849665 17382 log.cc:824] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:22.852205 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:22.849466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36409 (local address 127.0.0.1:46143)
0812 16:57:22.849659 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:22.849666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:22.850033 (+ 367us) server_negotiation.cc:407] Connection header received
0812 16:57:22.850219 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:22.850229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:22.850318 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:22.850438 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:22.851536 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:22.851541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:22.851544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:22.851559 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:22.851597 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:22.851613 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:22.851706 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:22.851854 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 16:57:22.851902 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 16:57:22.853375 17382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6: No bootstrap required, opened a new log
I20260812 16:57:22.860906 17382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } }
I20260812 16:57:22.861395 17382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:22.861521 17382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7e3f9d873c4d2d958e5a40ea15e0e6, State: Initialized, Role: FOLLOWER
I20260812 16:57:22.862141 17382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } }
I20260812 16:57:22.862381 17382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:22.862541 17382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:22.862746 17382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:22.865029 17382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } }
I20260812 16:57:22.865551 17382 leader_election.cc:302] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a7e3f9d873c4d2d958e5a40ea15e0e6; no voters:
I20260812 16:57:22.866159 17382 leader_election.cc:288] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:22.866319 17399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:22.867028 17399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a7e3f9d873c4d2d958e5a40ea15e0e6, State: Running, Role: LEADER
I20260812 16:57:22.867606 17399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } }
I20260812 16:57:22.868323 17382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:22.870501 17403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a7e3f9d873c4d2d958e5a40ea15e0e6. Latest consensus state: current_term: 1 leader_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } } }
I20260812 16:57:22.870932 17403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:22.871495 17402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7e3f9d873c4d2d958e5a40ea15e0e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46143 } } }
I20260812 16:57:22.871793 17402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:22.873306 17412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:22.878679 17412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:22.887913 17412 catalog_manager.cc:1347] Generated new cluster ID: 6b948f36d62749f5b8a2d95f60e432b2
I20260812 16:57:22.888028 17412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:22.917531 17412 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:22.918067 17412 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:22.939486 17412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a7e3f9d873c4d2d958e5a40ea15e0e6: Generated new TSK 0
I20260812 16:57:22.940670 17412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:22.946871 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17236
W20260812 16:57:22.948589 17369 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41455
W20260812 16:57:22.949332 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17236: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:22.949507 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17236
2026-08-12T16:57:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:23 dist-test-slave-z8x9 krb5kdc[17438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:23 dist-test-slave-z8x9 krb5kdc[17438](info): set up 2 sockets
Aug 12 16:57:23 dist-test-slave-z8x9 krb5kdc[17438](info): commencing operation
krb5kdc: starting...
W20260812 16:57:24.429826 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.424s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:24 dist-test-slave-z8x9 krb5kdc[17438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:24.776813 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:25.130491 17459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:25.130789 17459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:25.130982 17459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:25.138326 17459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:25.138442 17459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:25.138554 17459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:25.138664 17459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:25.138756 17459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:25.148690 17459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:25.150081 17459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:25.151461 17459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:25.164079 17477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:25.164203 17471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:25.164175 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:25.164908 17459 server_base.cc:1034] running on GCE node
I20260812 16:57:25.165876 17459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:25.167397 17459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:25.168644 17459 hybrid_clock.cc:648] HybridClock initialized: now 1786553845168603 us; error 52 us; skew 500 ppm
I20260812 16:57:25.169019 17459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:25 dist-test-slave-z8x9 krb5kdc[17438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:25.172501 17459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:25.174190 17459 webserver.cc:460] Webserver started at http://127.0.0.1:40339/ using document root <none> and password file <none>
I20260812 16:57:25.174968 17459 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:25.175125 17459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:25.175462 17459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:25.178742 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "b560024a00274344872f4200085b7078"
format_stamp: "Formatted at 2026-08-12 16:57:25 on dist-test-slave-z8x9"
server_key: "726b3b70f34c5a827cd68ff6936ae503"
server_key_iv: "5ed2da2705d2e305e18252bca5a76c27"
server_key_version: "encryptionkey@0"
I20260812 16:57:25.179565 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "b560024a00274344872f4200085b7078"
format_stamp: "Formatted at 2026-08-12 16:57:25 on dist-test-slave-z8x9"
server_key: "726b3b70f34c5a827cd68ff6936ae503"
server_key_iv: "5ed2da2705d2e305e18252bca5a76c27"
server_key_version: "encryptionkey@0"
I20260812 16:57:25.185034 17459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 16:57:25.188651 17493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:25.190112 17459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:57:25.190318 17459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "b560024a00274344872f4200085b7078"
format_stamp: "Formatted at 2026-08-12 16:57:25 on dist-test-slave-z8x9"
server_key: "726b3b70f34c5a827cd68ff6936ae503"
server_key_iv: "5ed2da2705d2e305e18252bca5a76c27"
server_key_version: "encryptionkey@0"
I20260812 16:57:25.190546 17459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:25.213121 17459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:25.218124 17459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:25.218465 17459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:25.238878 17459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35019
I20260812 16:57:25.238919 17608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35019 every 8 connection(s)
I20260812 16:57:25.240391 17459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:25.241837 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17459
I20260812 16:57:25.242039 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:25.242585 8974 external_mini_cluster.cc:1428] Setting key 5841115ad96670a856fca5dcb940cf29
I20260812 16:57:25.245225 17611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 16:57:25 dist-test-slave-z8x9 krb5kdc[17438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:57:25.259625 17611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078: Bootstrap starting.
I20260812 16:57:25.264235 17611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:25.265753 17611 log.cc:824] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:25.268858 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:25.244492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54601 (local address 127.0.0.1:35019)
0812 16:57:25.245400 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:25.245415 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:25.245470 (+ 55us) server_negotiation.cc:407] Connection header received
0812 16:57:25.247162 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:25.247186 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:25.247592 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:25.247952 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:25.249891 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:25.251284 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:25.253264 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:25.254025 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:25.254172 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:25.259417 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:25.259431 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:25.259453 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:25.259512 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:25.263753 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:25.264650 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:25.264672 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:25.264683 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:25.264926 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:25.265390 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:25.265397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:25.265402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:25.266191 (+ 789us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:25.266567 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:25.267053 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 16:57:25.267477 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":406,"threads_started":1}
I20260812 16:57:25.269552 17611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078: No bootstrap required, opened a new log
I20260812 16:57:25.272810 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:25.278688 17611 raft_consensus.cc:348] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } }
I20260812 16:57:25.279109 17611 raft_consensus.cc:374] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:25.279228 17611 raft_consensus.cc:729] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b560024a00274344872f4200085b7078, State: Initialized, Role: FOLLOWER
I20260812 16:57:25.279872 17611 consensus_queue.cc:260] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } }
I20260812 16:57:25.280112 17611 raft_consensus.cc:388] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:25.280277 17611 raft_consensus.cc:482] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:25.280489 17611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:25.282905 17611 raft_consensus.cc:504] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } }
I20260812 16:57:25.283450 17611 leader_election.cc:302] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b560024a00274344872f4200085b7078; no voters:
I20260812 16:57:25.284226 17611 leader_election.cc:288] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:25.284327 17626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:25.286682 17611 sys_catalog.cc:564] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:25.289584 17626 raft_consensus.cc:686] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [term 1 LEADER]: Becoming Leader. State: Replica: b560024a00274344872f4200085b7078, State: Running, Role: LEADER
I20260812 16:57:25.291237 17626 consensus_queue.cc:237] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } }
I20260812 16:57:25.294508 17627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b560024a00274344872f4200085b7078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } } }
I20260812 16:57:25.294939 17627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:25.295501 17630 sys_catalog.cc:455] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b560024a00274344872f4200085b7078. Latest consensus state: current_term: 1 leader_uuid: "b560024a00274344872f4200085b7078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b560024a00274344872f4200085b7078" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35019 } } }
I20260812 16:57:25.295817 17630 sys_catalog.cc:458] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:25.296520 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:25.277347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35653 (local address 127.0.0.1:35019)
0812 16:57:25.277542 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:25.277546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:25.278093 (+ 547us) server_negotiation.cc:407] Connection header received
0812 16:57:25.278298 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:25.278306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:25.278387 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:25.278498 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:25.280255 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:25.281024 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:25.282352 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:25.282810 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:25.282984 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:25.286293 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:25.286303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:25.286310 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:25.286380 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:25.291449 (+ 5069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:25.292484 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:25.292493 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:25.292500 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:25.292741 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:25.293435 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:25.293442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:25.293448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:25.293889 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:25.294260 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:25.295857 (+ 1597us) server_negotiation.cc:299] Negotiation successful
0812 16:57:25.296127 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 16:57:25.299672 17657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:25.304534 17657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:25.314152 17657 catalog_manager.cc:1347] Generated new cluster ID: a67ab330275b496890a184dcce19297b
I20260812 16:57:25.314308 17657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:25.345376 17657 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:25.346789 17657 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:25.354827 17657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b560024a00274344872f4200085b7078: Generated new TSK 0
I20260812 16:57:25.355821 17657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:25.368597 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17459
I20260812 16:57:25.370235 17594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:25.466536 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17459
2026-08-12T16:57:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:25 dist-test-slave-z8x9 krb5kdc[17678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:25 dist-test-slave-z8x9 krb5kdc[17678](info): set up 2 sockets
Aug 12 16:57:25 dist-test-slave-z8x9 krb5kdc[17678](info): commencing operation
krb5kdc: starting...
W20260812 16:57:26.921373 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.406s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:27 dist-test-slave-z8x9 krb5kdc[17678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:27.262118 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:27.610265 17700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:27.610579 17700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:27.610754 17700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:27.617988 17700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:27.618134 17700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:27.618256 17700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:27.618366 17700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:27.618446 17700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:27.628266 17700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:27.629654 17700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:27.630976 17700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:27.643136 17712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:27.643179 17711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:27.643095 17715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:27.643694 17700 server_base.cc:1034] running on GCE node
I20260812 16:57:27.644673 17700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:27.646169 17700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:27.647434 17700 hybrid_clock.cc:648] HybridClock initialized: now 1786553847647403 us; error 46 us; skew 500 ppm
I20260812 16:57:27.647805 17700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:27 dist-test-slave-z8x9 krb5kdc[17678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:27.651096 17700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:27.652652 17700 webserver.cc:460] Webserver started at http://127.0.0.1:36061/ using document root <none> and password file <none>
I20260812 16:57:27.653471 17700 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:27.653630 17700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:27.653956 17700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:27.657032 17700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "5686bfde2a974d12bd6183aaf120406c"
format_stamp: "Formatted at 2026-08-12 16:57:27 on dist-test-slave-z8x9"
server_key: "003d3a073b0b447eb6d08f81a98ad11f"
server_key_iv: "f3a1ab06ddb801494db10330b14acd20"
server_key_version: "encryptionkey@0"
I20260812 16:57:27.657925 17700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "5686bfde2a974d12bd6183aaf120406c"
format_stamp: "Formatted at 2026-08-12 16:57:27 on dist-test-slave-z8x9"
server_key: "003d3a073b0b447eb6d08f81a98ad11f"
server_key_iv: "f3a1ab06ddb801494db10330b14acd20"
server_key_version: "encryptionkey@0"
I20260812 16:57:27.663576 17700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:27.667232 17726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:27.668604 17700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:57:27.668799 17700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "5686bfde2a974d12bd6183aaf120406c"
format_stamp: "Formatted at 2026-08-12 16:57:27 on dist-test-slave-z8x9"
server_key: "003d3a073b0b447eb6d08f81a98ad11f"
server_key_iv: "f3a1ab06ddb801494db10330b14acd20"
server_key_version: "encryptionkey@0"
I20260812 16:57:27.669013 17700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:27.687237 17700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:27.691596 17700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:27.691908 17700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:27.711980 17700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39261
I20260812 16:57:27.712021 17820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39261 every 8 connection(s)
I20260812 16:57:27.713572 17700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:27.715466 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17700
I20260812 16:57:27.715667 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:27.716152 8974 external_mini_cluster.cc:1428] Setting key 2a17102d11216e549cfaa5ab83a0fb35
I20260812 16:57:27.718185 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:27.732080 17824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c: Bootstrap starting.
I20260812 16:57:27.732649 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:27.717988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48255 (local address 127.0.0.1:39261)
0812 16:57:27.721695 (+ 3707us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:27.721707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:27.721758 (+ 51us) server_negotiation.cc:407] Connection header received
0812 16:57:27.723854 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:27.723878 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:27.724299 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:27.724734 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:27.725350 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:27.726675 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:27.728022 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:27.728707 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:27.728946 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:27.730267 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:27.730275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:27.730289 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:27.730842 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:27.730920 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:27.730945 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:27.731088 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:27.731327 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 16:57:27.731601 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3492,"thread_start_us":3270,"threads_started":1}
I20260812 16:57:27.735491 17824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:27.736584 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:27.736749 17824 log.cc:824] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:27.739797 17824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c: No bootstrap required, opened a new log
I20260812 16:57:27.748286 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:27.741219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58279 (local address 127.0.0.1:39261)
0812 16:57:27.741441 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:27.741454 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:27.741869 (+ 415us) server_negotiation.cc:407] Connection header received
0812 16:57:27.742074 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:27.742077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:27.742141 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:27.742224 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:27.742918 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:27.743681 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:27.745672 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:27.746163 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:27.746269 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:27.747518 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:27.747524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:27.747527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:27.747544 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:27.747584 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:27.747602 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:27.747679 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:27.747852 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 16:57:27.747925 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:57:27.750293 17824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260812 16:57:27.750718 17824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:27.750834 17824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5686bfde2a974d12bd6183aaf120406c, State: Initialized, Role: FOLLOWER
I20260812 16:57:27.751467 17824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260812 16:57:27.751731 17824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:27.751886 17824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:27.752068 17824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:27.754925 17824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260812 16:57:27.755463 17824 leader_election.cc:302] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5686bfde2a974d12bd6183aaf120406c; no voters:
I20260812 16:57:27.756294 17824 leader_election.cc:288] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:27.756350 17848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:27.757275 17848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [term 1 LEADER]: Becoming Leader. State: Replica: 5686bfde2a974d12bd6183aaf120406c, State: Running, Role: LEADER
I20260812 16:57:27.757926 17848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } }
I20260812 16:57:27.758675 17824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:27.760790 17850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5686bfde2a974d12bd6183aaf120406c. Latest consensus state: current_term: 1 leader_uuid: "5686bfde2a974d12bd6183aaf120406c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } } }
I20260812 16:57:27.761260 17850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:27.761495 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5686bfde2a974d12bd6183aaf120406c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5686bfde2a974d12bd6183aaf120406c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39261 } } }
I20260812 16:57:27.761899 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:27.763878 17857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:27.770670 17857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:27.780294 17857 catalog_manager.cc:1347] Generated new cluster ID: e87f4e25fafe4d3395b36706ed798a96
I20260812 16:57:27.780453 17857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:27.789942 17857 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:27.791648 17857 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:27.805428 17857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5686bfde2a974d12bd6183aaf120406c: Generated new TSK 0
I20260812 16:57:27.806280 17857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:27.816637 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17700
W20260812 16:57:27.818032 17813 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48255
W20260812 16:57:27.818852 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17700: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:27.819015 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17700
2026-08-12T16:57:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:27 dist-test-slave-z8x9 krb5kdc[17886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:27 dist-test-slave-z8x9 krb5kdc[17886](info): set up 2 sockets
Aug 12 16:57:27 dist-test-slave-z8x9 krb5kdc[17886](info): commencing operation
krb5kdc: starting...
W20260812 16:57:29.259909 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.394s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:29 dist-test-slave-z8x9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:29.604135 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:29.954514 17907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:29.954845 17907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:29.955003 17907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:29.962626 17907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:29.962756 17907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:29.962855 17907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:29.962963 17907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:29.963037 17907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:29.973459 17907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:29.974896 17907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:29.976253 17907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:29.988279 17922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:29.988318 17916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:29.988359 17919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:29.989311 17907 server_base.cc:1034] running on GCE node
I20260812 16:57:29.990188 17907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:29.991823 17907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:29.993100 17907 hybrid_clock.cc:648] HybridClock initialized: now 1786553849993050 us; error 43 us; skew 500 ppm
I20260812 16:57:29.993578 17907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:29 dist-test-slave-z8x9 krb5kdc[17886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:30.002442 17907 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:30.004328 17907 webserver.cc:460] Webserver started at http://127.0.0.1:35589/ using document root <none> and password file <none>
I20260812 16:57:30.005123 17907 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:30.005343 17907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:30.005686 17907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:30.008924 17907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "71b0e484f6214e59a0b64c4761179fee"
format_stamp: "Formatted at 2026-08-12 16:57:30 on dist-test-slave-z8x9"
server_key: "24fc5d14c5f8372c1402f58631da9cd5"
server_key_iv: "00c9d0c0f977445693f012b9e63b0796"
server_key_version: "encryptionkey@0"
I20260812 16:57:30.009881 17907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "71b0e484f6214e59a0b64c4761179fee"
format_stamp: "Formatted at 2026-08-12 16:57:30 on dist-test-slave-z8x9"
server_key: "24fc5d14c5f8372c1402f58631da9cd5"
server_key_iv: "00c9d0c0f977445693f012b9e63b0796"
server_key_version: "encryptionkey@0"
I20260812 16:57:30.015825 17907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:57:30.019767 17935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:30.021540 17907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:57:30.021757 17907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "71b0e484f6214e59a0b64c4761179fee"
format_stamp: "Formatted at 2026-08-12 16:57:30 on dist-test-slave-z8x9"
server_key: "24fc5d14c5f8372c1402f58631da9cd5"
server_key_iv: "00c9d0c0f977445693f012b9e63b0796"
server_key_version: "encryptionkey@0"
I20260812 16:57:30.021986 17907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:30.028545 17907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:30.032995 17907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:30.033387 17907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:30.056347 17907 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34383
I20260812 16:57:30.056391 18028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34383 every 8 connection(s)
I20260812 16:57:30.057945 17907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:30.061900 18032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:30.067754 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 17907
I20260812 16:57:30.067974 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:30.068490 8974 external_mini_cluster.cc:1428] Setting key 0ed6773eefd21d063e28dfac1bf0b6ff
I20260812 16:57:30.073997 18032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee: Bootstrap starting.
I20260812 16:57:30.077109 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:30.070692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54927 (local address 127.0.0.1:34383)
0812 16:57:30.071656 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:30.071669 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:30.071712 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:57:30.073517 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:30.073542 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:30.073888 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:30.074326 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:30.075449 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:30.075457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:30.075470 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:30.075483 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:30.075544 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:30.075563 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:30.075742 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:30.075898 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 16:57:30.076105 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":427,"threads_started":1}
I20260812 16:57:30.077951 18032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:30.079061 18032 log.cc:824] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:30.081037 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:30.082275 18032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee: No bootstrap required, opened a new log
I20260812 16:57:30.088701 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:30.085839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38347 (local address 127.0.0.1:34383)
0812 16:57:30.086047 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:30.086053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:30.086611 (+ 558us) server_negotiation.cc:407] Connection header received
0812 16:57:30.086780 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:30.086784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:30.086839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:30.086929 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:30.087946 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:30.087950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:30.087954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:30.087969 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:30.088012 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:30.088028 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:30.088125 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:30.088266 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 16:57:30.088328 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 16:57:30.091539 18032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260812 16:57:30.092005 18032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:30.092123 18032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b0e484f6214e59a0b64c4761179fee, State: Initialized, Role: FOLLOWER
I20260812 16:57:30.092880 18032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260812 16:57:30.093322 18032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:30.093561 18032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:30.093850 18032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:30.096174 18032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260812 16:57:30.096786 18032 leader_election.cc:302] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71b0e484f6214e59a0b64c4761179fee; no voters:
I20260812 16:57:30.097541 18032 leader_election.cc:288] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:30.097704 18048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:30.098387 18048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [term 1 LEADER]: Becoming Leader. State: Replica: 71b0e484f6214e59a0b64c4761179fee, State: Running, Role: LEADER
I20260812 16:57:30.099017 18048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260812 16:57:30.099810 18032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:30.102274 18050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b0e484f6214e59a0b64c4761179fee. Latest consensus state: current_term: 1 leader_uuid: "71b0e484f6214e59a0b64c4761179fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } } }
I20260812 16:57:30.102263 18049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71b0e484f6214e59a0b64c4761179fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71b0e484f6214e59a0b64c4761179fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } } }
I20260812 16:57:30.102779 18050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:30.102808 18049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:30.106794 18058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:30.111641 18058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:30.121330 18058 catalog_manager.cc:1347] Generated new cluster ID: c0a41752967b49519a40ac9210d79f0d
I20260812 16:57:30.121480 18058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:30.150902 18058 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:30.151517 18058 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:30.172986 18058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71b0e484f6214e59a0b64c4761179fee: Generated new TSK 0
I20260812 16:57:30.174170 18058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:30.186352 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17907
W20260812 16:57:30.187731 18021 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54927
W20260812 16:57:30.188498 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 17907: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:30.188659 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 17907
2026-08-12T16:57:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:30 dist-test-slave-z8x9 krb5kdc[18083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:30 dist-test-slave-z8x9 krb5kdc[18083](info): set up 2 sockets
Aug 12 16:57:30 dist-test-slave-z8x9 krb5kdc[18083](info): commencing operation
krb5kdc: starting...
W20260812 16:57:31.624437 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:31 dist-test-slave-z8x9 krb5kdc[18083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:31.968999 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:32.322954 18105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:32.323272 18105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:32.323470 18105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:32.330905 18105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:32.331045 18105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:32.331167 18105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:32.331274 18105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:32.331362 18105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:32.341859 18105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:32.343297 18105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:32.344663 18105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:32.357735 18129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:32.357829 18121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:32.357764 18124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:32.358712 18105 server_base.cc:1034] running on GCE node
I20260812 16:57:32.359521 18105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:32.361016 18105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:32.362262 18105 hybrid_clock.cc:648] HybridClock initialized: now 1786553852362218 us; error 53 us; skew 500 ppm
I20260812 16:57:32.362651 18105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:32 dist-test-slave-z8x9 krb5kdc[18083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:32.366048 18105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:32.367547 18105 webserver.cc:460] Webserver started at http://127.0.0.1:43775/ using document root <none> and password file <none>
I20260812 16:57:32.368220 18105 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:32.368326 18105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:32.368590 18105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:32.371717 18105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1"
format_stamp: "Formatted at 2026-08-12 16:57:32 on dist-test-slave-z8x9"
server_key: "96bf87ce92349591081a7a82c2e4614a"
server_key_iv: "542840aaf7769ef0d2b2b591ffc6e428"
server_key_version: "encryptionkey@0"
I20260812 16:57:32.372467 18105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1"
format_stamp: "Formatted at 2026-08-12 16:57:32 on dist-test-slave-z8x9"
server_key: "96bf87ce92349591081a7a82c2e4614a"
server_key_iv: "542840aaf7769ef0d2b2b591ffc6e428"
server_key_version: "encryptionkey@0"
I20260812 16:57:32.378306 18105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 16:57:32.382020 18140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:32.383466 18105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:32.383662 18105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1"
format_stamp: "Formatted at 2026-08-12 16:57:32 on dist-test-slave-z8x9"
server_key: "96bf87ce92349591081a7a82c2e4614a"
server_key_iv: "542840aaf7769ef0d2b2b591ffc6e428"
server_key_version: "encryptionkey@0"
I20260812 16:57:32.383867 18105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:32.400161 18105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:32.404722 18105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:32.405036 18105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:32.425091 18105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46441
I20260812 16:57:32.425133 18252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46441 every 8 connection(s)
I20260812 16:57:32.426590 18105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:32.430392 18253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:32.433444 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18105
I20260812 16:57:32.433632 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:32.434211 8974 external_mini_cluster.cc:1428] Setting key bc95ade4b81ebfbb223050a8e8ce4b60
I20260812 16:57:32.444201 18253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1: Bootstrap starting.
I20260812 16:57:32.448537 18253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1: Neither blocks nor log segments found. Creating new log.
Aug 12 16:57:32 dist-test-slave-z8x9 krb5kdc[18083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:57:32.449867 18253 log.cc:824] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:32.453111 18253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1: No bootstrap required, opened a new log
I20260812 16:57:32.459043 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:32.436147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:59359 (local address 127.0.0.1:46441)
0812 16:57:32.437029 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:32.437041 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:32.437087 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:57:32.438903 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:32.438928 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:32.439356 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:32.439687 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:32.441770 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:32.443164 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:32.445010 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:32.445764 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:32.445918 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:32.451121 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:32.451130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:32.451145 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:32.451187 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:32.454588 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:32.455427 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:32.455434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:32.455441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:32.455622 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:32.456128 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:32.456132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:32.456134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:32.456699 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:32.457045 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:32.457636 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 16:57:32.457983 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":414,"threads_started":1}
I20260812 16:57:32.461814 18253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260812 16:57:32.462359 18253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:32.462494 18253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d44ffcb5ba4d419a6b03ca6892d2b1, State: Initialized, Role: FOLLOWER
I20260812 16:57:32.463189 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:32.463213 18253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260812 16:57:32.463514 18253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:32.463670 18253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:32.463845 18253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:32.466190 18253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260812 16:57:32.466718 18253 leader_election.cc:302] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d44ffcb5ba4d419a6b03ca6892d2b1; no voters:
I20260812 16:57:32.467387 18253 leader_election.cc:288] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:32.467555 18268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:32.469027 18268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [term 1 LEADER]: Becoming Leader. State: Replica: 32d44ffcb5ba4d419a6b03ca6892d2b1, State: Running, Role: LEADER
I20260812 16:57:32.469659 18268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260812 16:57:32.470176 18253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:32.472445 18273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d44ffcb5ba4d419a6b03ca6892d2b1. Latest consensus state: current_term: 1 leader_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260812 16:57:32.472873 18273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:32.473481 18269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d44ffcb5ba4d419a6b03ca6892d2b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260812 16:57:32.473814 18269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:32.484349 18280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:32.489481 18280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:32.499101 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:32.468586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:39221 (local address 127.0.0.1:46441)
0812 16:57:32.468773 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:32.468779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:32.480797 (+ 12018us) server_negotiation.cc:407] Connection header received
0812 16:57:32.481450 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:32.481460 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:32.481564 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:32.481641 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:32.483314 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:32.484388 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:32.486122 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:32.486954 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:32.487089 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:32.490447 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:32.490454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:32.490459 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:32.490500 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:32.494570 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:32.495569 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:32.495574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:32.495577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:32.495730 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:32.497003 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:32.497012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:32.497016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:32.497512 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:32.497778 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:32.498500 (+ 722us) server_negotiation.cc:299] Negotiation successful
0812 16:57:32.498707 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:57:32.499671 18280 catalog_manager.cc:1347] Generated new cluster ID: a82b1b89d16a4d8e81b5e374873da4ad
I20260812 16:57:32.499799 18280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:32.526891 18280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:32.528380 18280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:32.538563 18280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32d44ffcb5ba4d419a6b03ca6892d2b1: Generated new TSK 0
I20260812 16:57:32.539521 18280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:32.552937 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18105
I20260812 16:57:32.554507 18241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:32.650619 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18105
W20260812 16:57:32.680567 18078 connection.cc:535] client connection to 127.0.0.1:46441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46441): Connection reset by peer (error 104)
2026-08-12T16:57:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:32 dist-test-slave-z8x9 krb5kdc[18315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:32 dist-test-slave-z8x9 krb5kdc[18315](info): set up 2 sockets
Aug 12 16:57:32 dist-test-slave-z8x9 krb5kdc[18315](info): commencing operation
krb5kdc: starting...
W20260812 16:57:34.086035 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:34 dist-test-slave-z8x9 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:34.436218 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:34.785989 18343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:34.786298 18343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:34.786482 18343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:34.793905 18343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:34.794029 18343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:34.794134 18343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:34.794231 18343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:34.794312 18343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:34.804565 18343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:34.805965 18343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:34.807269 18343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:34.819401 18352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:34.819401 18351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:34.819437 18355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:34.820641 18343 server_base.cc:1034] running on GCE node
I20260812 16:57:34.821597 18343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:34.823042 18343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:34.824263 18343 hybrid_clock.cc:648] HybridClock initialized: now 1786553854824202 us; error 49 us; skew 500 ppm
I20260812 16:57:34.824659 18343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:34 dist-test-slave-z8x9 krb5kdc[18315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:34.828613 18343 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:34.830691 18343 webserver.cc:460] Webserver started at http://127.0.0.1:38687/ using document root <none> and password file <none>
I20260812 16:57:34.831420 18343 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:34.831559 18343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:34.831892 18343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:34.834945 18343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "1665e7b02387450a9f99608d0767013a"
format_stamp: "Formatted at 2026-08-12 16:57:34 on dist-test-slave-z8x9"
server_key: "dd3a0ed268f1f2b4bf65862222a3ea02"
server_key_iv: "9ec9dfa40b084ce0bc12715e5a0e3e1e"
server_key_version: "encryptionkey@0"
I20260812 16:57:34.835728 18343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "1665e7b02387450a9f99608d0767013a"
format_stamp: "Formatted at 2026-08-12 16:57:34 on dist-test-slave-z8x9"
server_key: "dd3a0ed268f1f2b4bf65862222a3ea02"
server_key_iv: "9ec9dfa40b084ce0bc12715e5a0e3e1e"
server_key_version: "encryptionkey@0"
I20260812 16:57:34.841234 18343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 16:57:34.844705 18369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:34.846263 18343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:57:34.846477 18343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "1665e7b02387450a9f99608d0767013a"
format_stamp: "Formatted at 2026-08-12 16:57:34 on dist-test-slave-z8x9"
server_key: "dd3a0ed268f1f2b4bf65862222a3ea02"
server_key_iv: "9ec9dfa40b084ce0bc12715e5a0e3e1e"
server_key_version: "encryptionkey@0"
I20260812 16:57:34.846685 18343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:34.868145 18343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:34.872328 18343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:34.872635 18343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:34.891917 18343 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34493
I20260812 16:57:34.891954 18467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34493 every 8 connection(s)
I20260812 16:57:34.893348 18343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:34.897053 18470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:34.899199 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18343
I20260812 16:57:34.899395 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:34.899825 8974 external_mini_cluster.cc:1428] Setting key f71024f842dbd89e954fac080889c028
I20260812 16:57:34.909065 18470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a: Bootstrap starting.
I20260812 16:57:34.912808 18470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:34.913753 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:34.901627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:39223 (local address 127.0.0.1:34493)
0812 16:57:34.902564 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:34.902575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:34.902620 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:57:34.904294 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:34.904315 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:34.904685 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:34.904980 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:34.905629 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:34.906980 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:34.908839 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:34.909751 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:34.909903 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:34.911321 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:34.911332 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:34.911345 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:34.911887 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:34.911966 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:34.911991 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:34.912115 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:34.912429 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 16:57:34.912613 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":438,"threads_started":1}
I20260812 16:57:34.914084 18470 log.cc:824] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:34.917272 18470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a: No bootstrap required, opened a new log
I20260812 16:57:34.917940 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:34.925560 18470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } }
I20260812 16:57:34.925971 18470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:34.926087 18470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1665e7b02387450a9f99608d0767013a, State: Initialized, Role: FOLLOWER
I20260812 16:57:34.926729 18470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } }
I20260812 16:57:34.927008 18470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:34.927228 18470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:34.927424 18470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:34.929026 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:34.922542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:57577 (local address 127.0.0.1:34493)
0812 16:57:34.922726 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:34.922735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:34.923167 (+ 432us) server_negotiation.cc:407] Connection header received
0812 16:57:34.923416 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:34.923420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:34.923485 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:34.923602 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:34.924121 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:34.924934 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:34.926479 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:34.926924 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:34.927075 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:34.928249 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:34.928254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:34.928257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:34.928273 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:34.928330 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:34.928348 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:34.928433 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:34.928616 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 16:57:34.928668 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 16:57:34.930670 18470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } }
I20260812 16:57:34.931233 18470 leader_election.cc:302] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1665e7b02387450a9f99608d0767013a; no voters:
I20260812 16:57:34.931797 18470 leader_election.cc:288] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:34.931943 18487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:34.932749 18487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [term 1 LEADER]: Becoming Leader. State: Replica: 1665e7b02387450a9f99608d0767013a, State: Running, Role: LEADER
I20260812 16:57:34.933333 18487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } }
I20260812 16:57:34.933964 18470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:34.936430 18489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1665e7b02387450a9f99608d0767013a. Latest consensus state: current_term: 1 leader_uuid: "1665e7b02387450a9f99608d0767013a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } } }
I20260812 16:57:34.936904 18489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:34.938041 18488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1665e7b02387450a9f99608d0767013a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1665e7b02387450a9f99608d0767013a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34493 } } }
I20260812 16:57:34.939088 18488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:34.942132 18506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:34.947571 18506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:34.956787 18506 catalog_manager.cc:1347] Generated new cluster ID: fd22a84a58d64213ba754c3939e1a0dd
I20260812 16:57:34.956938 18506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:34.980825 18506 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:34.982316 18506 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:34.999501 18506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1665e7b02387450a9f99608d0767013a: Generated new TSK 0
I20260812 16:57:35.000490 18506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:35.015463 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18343
W20260812 16:57:35.016767 18461 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:39223
W20260812 16:57:35.017741 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18343: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:35.017885 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18343
W20260812 16:57:35.042659 18309 connection.cc:535] client connection to 127.0.0.1:34493 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34493): Connection reset by peer (error 104)
2026-08-12T16:57:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:35 dist-test-slave-z8x9 krb5kdc[18539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:35 dist-test-slave-z8x9 krb5kdc[18539](info): set up 2 sockets
Aug 12 16:57:35 dist-test-slave-z8x9 krb5kdc[18539](info): commencing operation
krb5kdc: starting...
W20260812 16:57:36.453914 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.392s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:36 dist-test-slave-z8x9 krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:36.802008 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:37.153472 18565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:37.153877 18565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:37.153995 18565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:37.161404 18565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:37.161535 18565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:37.161587 18565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:37.161650 18565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:37.161696 18565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:37.171868 18565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:37.173141 18565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:37.174561 18565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:37.186730 18573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:37.186766 18577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:37.186730 18574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:37.187737 18565 server_base.cc:1034] running on GCE node
I20260812 16:57:37.188553 18565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:37.190056 18565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:37.191298 18565 hybrid_clock.cc:648] HybridClock initialized: now 1786553857191253 us; error 35 us; skew 500 ppm
I20260812 16:57:37.191660 18565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:37 dist-test-slave-z8x9 krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:37.195060 18565 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:37.196679 18565 webserver.cc:460] Webserver started at http://127.0.0.1:44815/ using document root <none> and password file <none>
I20260812 16:57:37.197497 18565 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:37.197654 18565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:37.197980 18565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:37.201216 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "e785c88968d9416e8278790567a6ed02"
format_stamp: "Formatted at 2026-08-12 16:57:37 on dist-test-slave-z8x9"
server_key: "66aef17342eedb6c38a2d7fd41515448"
server_key_iv: "ad7a64acf274aa81482c6b4a91c3bf24"
server_key_version: "encryptionkey@0"
I20260812 16:57:37.202044 18565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "e785c88968d9416e8278790567a6ed02"
format_stamp: "Formatted at 2026-08-12 16:57:37 on dist-test-slave-z8x9"
server_key: "66aef17342eedb6c38a2d7fd41515448"
server_key_iv: "ad7a64acf274aa81482c6b4a91c3bf24"
server_key_version: "encryptionkey@0"
I20260812 16:57:37.207456 18565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 16:57:37.211001 18587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:37.212350 18565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 16:57:37.212543 18565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "e785c88968d9416e8278790567a6ed02"
format_stamp: "Formatted at 2026-08-12 16:57:37 on dist-test-slave-z8x9"
server_key: "66aef17342eedb6c38a2d7fd41515448"
server_key_iv: "ad7a64acf274aa81482c6b4a91c3bf24"
server_key_version: "encryptionkey@0"
I20260812 16:57:37.212751 18565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:37.218966 18565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:37.223520 18565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:37.223842 18565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:37.243606 18565 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41971
I20260812 16:57:37.243662 18694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41971 every 8 connection(s)
I20260812 16:57:37.245085 18565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:37.245568 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18565
I20260812 16:57:37.245771 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:37.246373 8974 external_mini_cluster.cc:1428] Setting key 4c84db5968c4f1461288fdd76b7b7e62
I20260812 16:57:37.249738 18702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:37.254561 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:37.248310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:60049 (local address 127.0.0.1:41971)
0812 16:57:37.249082 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:37.249093 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:37.249136 (+ 43us) server_negotiation.cc:407] Connection header received
0812 16:57:37.250927 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:37.250951 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:37.251296 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:37.251688 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:37.252834 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:37.252844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:37.252860 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:37.252883 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:37.252954 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:37.252975 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:37.253264 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:37.253416 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 16:57:37.253610 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":325,"threads_started":1}
I20260812 16:57:37.258677 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:37.261888 18702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02: Bootstrap starting.
I20260812 16:57:37.265386 18702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:37.265910 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:37.263356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:46105 (local address 127.0.0.1:41971)
0812 16:57:37.263571 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:37.263580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:37.263834 (+ 254us) server_negotiation.cc:407] Connection header received
0812 16:57:37.264029 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:37.264035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:37.264087 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:37.264186 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:37.265219 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:37.265224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:37.265232 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:37.265247 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:37.265296 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:37.265311 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:37.265442 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:37.265572 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 16:57:37.265636 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 16:57:37.266847 18702 log.cc:824] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:37.269737 18702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02: No bootstrap required, opened a new log
I20260812 16:57:37.277384 18702 raft_consensus.cc:348] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } }
I20260812 16:57:37.277767 18702 raft_consensus.cc:374] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:37.277839 18702 raft_consensus.cc:729] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e785c88968d9416e8278790567a6ed02, State: Initialized, Role: FOLLOWER
I20260812 16:57:37.278518 18702 consensus_queue.cc:260] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } }
I20260812 16:57:37.278842 18702 raft_consensus.cc:388] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:37.279035 18702 raft_consensus.cc:482] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:37.279219 18702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:37.281535 18702 raft_consensus.cc:504] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } }
I20260812 16:57:37.282123 18702 leader_election.cc:302] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e785c88968d9416e8278790567a6ed02; no voters:
I20260812 16:57:37.282810 18702 leader_election.cc:288] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:37.282997 18722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:37.283919 18722 raft_consensus.cc:686] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [term 1 LEADER]: Becoming Leader. State: Replica: e785c88968d9416e8278790567a6ed02, State: Running, Role: LEADER
I20260812 16:57:37.284495 18722 consensus_queue.cc:237] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } }
I20260812 16:57:37.285033 18702 sys_catalog.cc:564] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:37.287642 18725 sys_catalog.cc:455] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e785c88968d9416e8278790567a6ed02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } } }
I20260812 16:57:37.287611 18726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e785c88968d9416e8278790567a6ed02. Latest consensus state: current_term: 1 leader_uuid: "e785c88968d9416e8278790567a6ed02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785c88968d9416e8278790567a6ed02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41971 } } }
I20260812 16:57:37.288110 18725 sys_catalog.cc:458] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:37.288141 18726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:37.291231 18735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:37.296685 18735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:37.305855 18735 catalog_manager.cc:1347] Generated new cluster ID: 9fdf33d5ef474e5297e5efb262443754
I20260812 16:57:37.305998 18735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:37.319299 18735 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:37.319716 18735 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:37.325253 18735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e785c88968d9416e8278790567a6ed02: Generated new TSK 0
I20260812 16:57:37.326086 18735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:37.342322 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18565
W20260812 16:57:37.343621 18688 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:60049
W20260812 16:57:37.344300 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18565: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:37.344463 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18565
2026-08-12T16:57:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:37 dist-test-slave-z8x9 krb5kdc[18759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:37 dist-test-slave-z8x9 krb5kdc[18759](info): set up 2 sockets
Aug 12 16:57:37 dist-test-slave-z8x9 krb5kdc[18759](info): commencing operation
krb5kdc: starting...
W20260812 16:57:38.777580 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:39.125414 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:39.489101 18779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:39.489424 18779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:39.489614 18779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:39.497453 18779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:39.497587 18779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:39.497673 18779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:39.497785 18779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:39.497869 18779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:39.507781 18779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:39.509227 18779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:39.510530 18779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:39.522146 18786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:39.522192 18790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:39.522240 18788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:39.523481 18779 server_base.cc:1034] running on GCE node
I20260812 16:57:39.524346 18779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:39.525801 18779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:39.527004 18779 hybrid_clock.cc:648] HybridClock initialized: now 1786553859526960 us; error 33 us; skew 500 ppm
I20260812 16:57:39.527318 18779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:39.530560 18779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:39.531960 18779 webserver.cc:460] Webserver started at http://127.0.0.1:35391/ using document root <none> and password file <none>
I20260812 16:57:39.532608 18779 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:39.532711 18779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:39.532985 18779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:39.536164 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "0bbba019b3404ff99895a6d341f8cacc"
format_stamp: "Formatted at 2026-08-12 16:57:39 on dist-test-slave-z8x9"
server_key: "0b9c93f01c3db499daab45b77b5fc4f3"
server_key_iv: "517e92c1e96968bb877e636586bfbfa3"
server_key_version: "encryptionkey@0"
I20260812 16:57:39.537017 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "0bbba019b3404ff99895a6d341f8cacc"
format_stamp: "Formatted at 2026-08-12 16:57:39 on dist-test-slave-z8x9"
server_key: "0b9c93f01c3db499daab45b77b5fc4f3"
server_key_iv: "517e92c1e96968bb877e636586bfbfa3"
server_key_version: "encryptionkey@0"
I20260812 16:57:39.542582 18779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:57:39.546240 18813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:39.547569 18779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:39.547780 18779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "0bbba019b3404ff99895a6d341f8cacc"
format_stamp: "Formatted at 2026-08-12 16:57:39 on dist-test-slave-z8x9"
server_key: "0b9c93f01c3db499daab45b77b5fc4f3"
server_key_iv: "517e92c1e96968bb877e636586bfbfa3"
server_key_version: "encryptionkey@0"
I20260812 16:57:39.548005 18779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:39.571000 18779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:39.575353 18779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:39.575660 18779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:39.595135 18779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36477
I20260812 16:57:39.595166 18916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36477 every 8 connection(s)
I20260812 16:57:39.596506 18779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:39.600219 18918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:39.600893 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 18779
I20260812 16:57:39.601099 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:39.601578 8974 external_mini_cluster.cc:1428] Setting key 21b6b9da36179eb3f0816f9d5175eed9
I20260812 16:57:39.614873 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc: Bootstrap starting.
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 16:57:39.618047 18918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:39.619477 18918 log.cc:824] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:39.623266 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc: No bootstrap required, opened a new log
I20260812 16:57:39.627957 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:39.603571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:44269 (local address 127.0.0.1:36477)
0812 16:57:39.604402 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:39.604418 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:39.604471 (+ 53us) server_negotiation.cc:407] Connection header received
0812 16:57:39.606976 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:39.607005 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:39.607542 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:39.608011 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:39.609467 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:39.611531 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:39.612910 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:39.614017 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:39.614211 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:39.618587 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:39.618600 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:39.618626 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:39.618676 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:39.622894 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:39.623797 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:39.623807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:39.623824 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:39.624041 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:39.624548 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:39.624553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:39.624557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:39.625318 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:39.625672 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:39.626139 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 16:57:39.626498 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":356,"threads_started":1}
I20260812 16:57:39.632073 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:39.633447 18918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } }
I20260812 16:57:39.633814 18918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:39.633921 18918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbba019b3404ff99895a6d341f8cacc, State: Initialized, Role: FOLLOWER
I20260812 16:57:39.634570 18918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } }
I20260812 16:57:39.634824 18918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:39.634989 18918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:39.635178 18918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:39.638235 18918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } }
I20260812 16:57:39.638918 18918 leader_election.cc:302] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bbba019b3404ff99895a6d341f8cacc; no voters:
I20260812 16:57:39.639633 18918 leader_election.cc:288] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:39.639683 18938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:39.641458 18938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [term 1 LEADER]: Becoming Leader. State: Replica: 0bbba019b3404ff99895a6d341f8cacc, State: Running, Role: LEADER
I20260812 16:57:39.642073 18918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:39.642197 18938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } }
I20260812 16:57:39.646581 18942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bbba019b3404ff99895a6d341f8cacc. Latest consensus state: current_term: 1 leader_uuid: "0bbba019b3404ff99895a6d341f8cacc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } } }
I20260812 16:57:39.647024 18942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:39.646849 18941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bbba019b3404ff99895a6d341f8cacc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbba019b3404ff99895a6d341f8cacc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36477 } } }
I20260812 16:57:39.647169 18941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:39.648196 18963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:39.655638 18963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:39.660118 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:39.636789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:52469 (local address 127.0.0.1:36477)
0812 16:57:39.637036 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:39.637042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:39.637429 (+ 387us) server_negotiation.cc:407] Connection header received
0812 16:57:39.637588 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:39.637591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:39.637651 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:39.637750 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:39.639842 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:39.641529 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:39.643511 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:39.644459 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:39.644623 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:39.648321 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:39.648330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:39.648337 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:39.648388 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:39.655025 (+ 6637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:39.656202 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:39.656207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:39.656210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:39.656397 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:39.657079 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:39.657085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:39.657091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:39.657626 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:39.659176 (+ 1550us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:39.659371 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 16:57:39.659550 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 16:57:39.669147 18963 catalog_manager.cc:1347] Generated new cluster ID: 2b050d799fb14806ac6c708957e41851
I20260812 16:57:39.669355 18963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:39.689697 18963 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:39.691051 18963 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:39.698589 18963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bbba019b3404ff99895a6d341f8cacc: Generated new TSK 0
I20260812 16:57:39.699390 18963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:39.704929 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 18779
I20260812 16:57:39.706301 18912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:39.797509 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 18779
W20260812 16:57:39.825860 18756 connection.cc:535] client connection to 127.0.0.1:36477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36477): Connection reset by peer (error 104)
2026-08-12T16:57:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18980](info): set up 2 sockets
Aug 12 16:57:39 dist-test-slave-z8x9 krb5kdc[18980](info): commencing operation
krb5kdc: starting...
W20260812 16:57:41.229069 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.009s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:41 dist-test-slave-z8x9 krb5kdc[18980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:41.582208 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:41.938364 19008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:41.938768 19008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:41.938951 19008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:41.946730 19008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:41.946904 19008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:41.947008 19008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:41.947115 19008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:41.947192 19008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:41.957422 19008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:41.958824 19008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:41.960167 19008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:41.972833 19019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:41.972891 19018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:41.973054 19022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:41.973812 19008 server_base.cc:1034] running on GCE node
I20260812 16:57:41.974910 19008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:41.976629 19008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:41.977916 19008 hybrid_clock.cc:648] HybridClock initialized: now 1786553861977871 us; error 57 us; skew 500 ppm
I20260812 16:57:41.978372 19008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:41 dist-test-slave-z8x9 krb5kdc[18980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:41.985787 19008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:41.987517 19008 webserver.cc:460] Webserver started at http://127.0.0.1:46825/ using document root <none> and password file <none>
I20260812 16:57:41.988255 19008 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:41.988416 19008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:41.988746 19008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:41.991976 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "d12c8fc3d19247d7b24372926eb67060"
format_stamp: "Formatted at 2026-08-12 16:57:41 on dist-test-slave-z8x9"
server_key: "d574eda83d9ff11ca9bed663ec77d5d9"
server_key_iv: "ae103a33d058b30d5e9af5cf92d2b854"
server_key_version: "encryptionkey@0"
I20260812 16:57:41.992846 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "d12c8fc3d19247d7b24372926eb67060"
format_stamp: "Formatted at 2026-08-12 16:57:41 on dist-test-slave-z8x9"
server_key: "d574eda83d9ff11ca9bed663ec77d5d9"
server_key_iv: "ae103a33d058b30d5e9af5cf92d2b854"
server_key_version: "encryptionkey@0"
I20260812 16:57:41.998673 19008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:57:42.002406 19030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:42.003858 19008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:42.004082 19008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "d12c8fc3d19247d7b24372926eb67060"
format_stamp: "Formatted at 2026-08-12 16:57:41 on dist-test-slave-z8x9"
server_key: "d574eda83d9ff11ca9bed663ec77d5d9"
server_key_iv: "ae103a33d058b30d5e9af5cf92d2b854"
server_key_version: "encryptionkey@0"
I20260812 16:57:42.004325 19008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:42.025375 19008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:42.030208 19008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:42.030514 19008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:42.050532 19008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46359
I20260812 16:57:42.050565 19129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46359 every 8 connection(s)
I20260812 16:57:42.051977 19008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:42.055902 19131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:42.056710 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19008
I20260812 16:57:42.056931 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:42.057606 8974 external_mini_cluster.cc:1428] Setting key ff5ec78217b5db368394fc49c65dfff3
I20260812 16:57:42.070410 19131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060: Bootstrap starting.
I20260812 16:57:42.071920 19135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:42.059592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:44309 (local address 127.0.0.1:46359)
0812 16:57:42.060520 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:42.060532 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:42.060605 (+ 73us) server_negotiation.cc:407] Connection header received
0812 16:57:42.062528 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:42.062555 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:42.062963 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:42.063301 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:42.063960 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:42.065317 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:42.067364 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:42.068097 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:42.068257 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:42.069576 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:42.069584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:42.069598 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:42.070151 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:42.070232 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:42.070257 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:42.070381 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:42.070754 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 16:57:42.070952 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":371,"threads_started":1}
I20260812 16:57:42.073758 19131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:42.074972 19131 log.cc:824] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:42.075726 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:42.077845 19131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060: No bootstrap required, opened a new log
I20260812 16:57:42.087415 19135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:42.080305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:57857 (local address 127.0.0.1:46359)
0812 16:57:42.080512 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:42.080518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:42.080795 (+ 277us) server_negotiation.cc:407] Connection header received
0812 16:57:42.080967 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:42.080971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:42.081034 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:42.081109 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:42.081750 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:42.082531 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:42.084502 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:42.084953 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:42.085057 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:42.086480 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:42.086493 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:42.086500 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:42.086524 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:42.086580 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:42.086605 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:42.086730 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:42.086956 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 16:57:42.087015 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 16:57:42.087684 19131 raft_consensus.cc:348] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } }
I20260812 16:57:42.088230 19131 raft_consensus.cc:374] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:42.088356 19131 raft_consensus.cc:729] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12c8fc3d19247d7b24372926eb67060, State: Initialized, Role: FOLLOWER
I20260812 16:57:42.089066 19131 consensus_queue.cc:260] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } }
I20260812 16:57:42.089389 19131 raft_consensus.cc:388] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:42.089499 19131 raft_consensus.cc:482] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:42.089633 19131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:42.091900 19131 raft_consensus.cc:504] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } }
I20260812 16:57:42.092360 19131 leader_election.cc:302] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d12c8fc3d19247d7b24372926eb67060; no voters:
I20260812 16:57:42.092944 19131 leader_election.cc:288] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:42.093101 19148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:42.094703 19148 raft_consensus.cc:686] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [term 1 LEADER]: Becoming Leader. State: Replica: d12c8fc3d19247d7b24372926eb67060, State: Running, Role: LEADER
I20260812 16:57:42.095284 19148 consensus_queue.cc:237] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } }
I20260812 16:57:42.095618 19131 sys_catalog.cc:564] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:42.098268 19152 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d12c8fc3d19247d7b24372926eb67060. Latest consensus state: current_term: 1 leader_uuid: "d12c8fc3d19247d7b24372926eb67060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } } }
I20260812 16:57:42.098794 19152 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:42.098404 19150 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d12c8fc3d19247d7b24372926eb67060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c8fc3d19247d7b24372926eb67060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46359 } } }
I20260812 16:57:42.099378 19150 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:42.100944 19162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:42.106436 19162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:42.115391 19162 catalog_manager.cc:1347] Generated new cluster ID: d487304557bc42428ee0c58e154fc9fe
I20260812 16:57:42.115522 19162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:42.151234 19162 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:42.152568 19162 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:42.170918 19162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d12c8fc3d19247d7b24372926eb67060: Generated new TSK 0
I20260812 16:57:42.171852 19162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:42.190691 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19008
W20260812 16:57:42.191884 19120 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:44309
W20260812 16:57:42.192871 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19008: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:42.193032 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19008
W20260812 16:57:42.223884 18977 connection.cc:535] client connection to 127.0.0.1:46359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46359): Connection reset by peer (error 104)
2026-08-12T16:57:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:42 dist-test-slave-z8x9 krb5kdc[19183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:42 dist-test-slave-z8x9 krb5kdc[19183](info): set up 2 sockets
Aug 12 16:57:42 dist-test-slave-z8x9 krb5kdc[19183](info): commencing operation
krb5kdc: starting...
W20260812 16:57:43.626788 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:43 dist-test-slave-z8x9 krb5kdc[19183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 16:57:43.980508 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:44.337021 19214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:44.337424 19214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:44.337610 19214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:44.344952 19214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:44.345098 19214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:44.345222 19214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:44.345325 19214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:44.345412 19214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:44.355602 19214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34301
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:44.357009 19214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:44.358413 19214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:44.370640 19222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:44.370659 19223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:44.371313 19227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:44.371435 19214 server_base.cc:1034] running on GCE node
I20260812 16:57:44.372316 19214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:44.373835 19214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:44.375082 19214 hybrid_clock.cc:648] HybridClock initialized: now 1786553864375020 us; error 53 us; skew 500 ppm
I20260812 16:57:44.375483 19214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:44 dist-test-slave-z8x9 krb5kdc[19183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:44.378952 19214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 16:57:44.380405 19214 webserver.cc:460] Webserver started at http://127.0.0.1:45155/ using document root <none> and password file <none>
I20260812 16:57:44.381057 19214 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:44.381161 19214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:44.381508 19214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:44.384572 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "4b9195efe7784159af9590c8ff914290"
format_stamp: "Formatted at 2026-08-12 16:57:44 on dist-test-slave-z8x9"
server_key: "90dabc55456fb33359bb64fa13d8c459"
server_key_iv: "dfdc6a7a4db80155a25fed1a8a214eac"
server_key_version: "encryptionkey@0"
I20260812 16:57:44.385406 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "4b9195efe7784159af9590c8ff914290"
format_stamp: "Formatted at 2026-08-12 16:57:44 on dist-test-slave-z8x9"
server_key: "90dabc55456fb33359bb64fa13d8c459"
server_key_iv: "dfdc6a7a4db80155a25fed1a8a214eac"
server_key_version: "encryptionkey@0"
I20260812 16:57:44.390678 19214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:57:44.394380 19238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:44.395835 19214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 16:57:44.396047 19214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "4b9195efe7784159af9590c8ff914290"
format_stamp: "Formatted at 2026-08-12 16:57:44 on dist-test-slave-z8x9"
server_key: "90dabc55456fb33359bb64fa13d8c459"
server_key_iv: "dfdc6a7a4db80155a25fed1a8a214eac"
server_key_version: "encryptionkey@0"
I20260812 16:57:44.396274 19214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:44.403558 19214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:44.408169 19214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:44.408536 19214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:44.427934 19214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34301
I20260812 16:57:44.427983 19337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34301 every 8 connection(s)
I20260812 16:57:44.429478 19214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:44.433480 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:44.437918 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19214
I20260812 16:57:44.438123 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:44.438581 8974 external_mini_cluster.cc:1428] Setting key baf0967f6f45991973914ed039f2ee73
I20260812 16:57:44.444887 19338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290: Bootstrap starting.
I20260812 16:57:44.446884 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:44.440338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:35929 (local address 127.0.0.1:34301)
0812 16:57:44.441121 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:44.441133 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:44.441234 (+ 101us) server_negotiation.cc:407] Connection header received
0812 16:57:44.443148 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:44.443171 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:44.443509 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:44.443823 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:44.445014 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:44.445021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:44.445034 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:44.445088 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:44.445281 (+ 193us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:44.445323 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:44.445557 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:44.445678 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 16:57:44.445903 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":346,"threads_started":1}
I20260812 16:57:44.448652 19338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:44.449963 19338 log.cc:824] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:44.450989 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:44.453378 19338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290: No bootstrap required, opened a new log
I20260812 16:57:44.459465 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:44.456617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:44583 (local address 127.0.0.1:34301)
0812 16:57:44.456836 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:44.456843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:44.457142 (+ 299us) server_negotiation.cc:407] Connection header received
0812 16:57:44.457397 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:44.457407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:44.457477 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:44.457590 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:44.458675 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:44.458680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:44.458685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:44.458710 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:44.458762 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:44.458788 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:44.458908 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:44.459061 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 16:57:44.459140 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 16:57:44.462888 19338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260812 16:57:44.463292 19338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:44.463420 19338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9195efe7784159af9590c8ff914290, State: Initialized, Role: FOLLOWER
I20260812 16:57:44.464016 19338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260812 16:57:44.464278 19338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:44.464447 19338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:44.464625 19338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:44.466986 19338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260812 16:57:44.467535 19338 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9195efe7784159af9590c8ff914290; no voters:
I20260812 16:57:44.468139 19338 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:44.468277 19360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:44.469013 19360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9195efe7784159af9590c8ff914290, State: Running, Role: LEADER
I20260812 16:57:44.469655 19360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } }
I20260812 16:57:44.470451 19338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:44.472865 19364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9195efe7784159af9590c8ff914290. Latest consensus state: current_term: 1 leader_uuid: "4b9195efe7784159af9590c8ff914290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } } }
I20260812 16:57:44.472774 19363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9195efe7784159af9590c8ff914290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9195efe7784159af9590c8ff914290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34301 } } }
I20260812 16:57:44.473203 19363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:44.473322 19364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:44.475101 19370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:44.480327 19370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:44.490382 19370 catalog_manager.cc:1347] Generated new cluster ID: 3e9a4f20c17241f2b2733505ff787e1b
I20260812 16:57:44.490510 19370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:44.522606 19370 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:44.523159 19370 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:44.532344 19370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9195efe7784159af9590c8ff914290: Generated new TSK 0
I20260812 16:57:44.533507 19370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:44.544443 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19214
W20260812 16:57:44.546083 19330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:35929
W20260812 16:57:44.546902 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19214: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:44.547070 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19214
2026-08-12T16:57:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:44 dist-test-slave-z8x9 krb5kdc[19394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:44 dist-test-slave-z8x9 krb5kdc[19394](info): set up 2 sockets
Aug 12 16:57:44 dist-test-slave-z8x9 krb5kdc[19394](info): commencing operation
krb5kdc: starting...
W20260812 16:57:46.067168 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.468s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:46 dist-test-slave-z8x9 krb5kdc[19394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:46.433527 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:32957
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:32957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:46.794263 19416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:46.794556 19416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:46.794746 19416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:46.801975 19416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:46.802096 19416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:46.802211 19416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:46.802290 19416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:46.802390 19416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:46.812314 19416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:32957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:32957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:46.813727 19416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:46.815068 19416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:46.826831 19423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:46.826925 19425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:46.826901 19429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:46.827750 19416 server_base.cc:1034] running on GCE node
I20260812 16:57:46.828594 19416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:46.829983 19416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:46.831226 19416 hybrid_clock.cc:648] HybridClock initialized: now 1786553866831182 us; error 51 us; skew 500 ppm
I20260812 16:57:46.831598 19416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:46 dist-test-slave-z8x9 krb5kdc[19394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553866, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:46.834717 19416 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:46.836081 19416 webserver.cc:460] Webserver started at http://127.8.195.190:43767/ using document root <none> and password file <none>
I20260812 16:57:46.836690 19416 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:46.836802 19416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:46.837045 19416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:46.840137 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "c5c6d68b13a74544bcc0a4786845cc01"
format_stamp: "Formatted at 2026-08-12 16:57:46 on dist-test-slave-z8x9"
server_key: "500313613939b10ad320db09b9a5f7c3"
server_key_iv: "e4364ddd121e981179c10fc717a1e0da"
server_key_version: "encryptionkey@0"
I20260812 16:57:46.840888 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "c5c6d68b13a74544bcc0a4786845cc01"
format_stamp: "Formatted at 2026-08-12 16:57:46 on dist-test-slave-z8x9"
server_key: "500313613939b10ad320db09b9a5f7c3"
server_key_iv: "e4364ddd121e981179c10fc717a1e0da"
server_key_version: "encryptionkey@0"
I20260812 16:57:46.846064 19416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 16:57:46.849351 19448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:46.850672 19416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:57:46.850808 19416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "c5c6d68b13a74544bcc0a4786845cc01"
format_stamp: "Formatted at 2026-08-12 16:57:46 on dist-test-slave-z8x9"
server_key: "500313613939b10ad320db09b9a5f7c3"
server_key_iv: "e4364ddd121e981179c10fc717a1e0da"
server_key_version: "encryptionkey@0"
I20260812 16:57:46.850975 19416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:46.877589 19416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:46.881856 19416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:46.882180 19416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:46.901773 19416 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:32957
I20260812 16:57:46.901793 19553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:32957 every 8 connection(s)
I20260812 16:57:46.903244 19416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:46.907091 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:46.913141 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19416
I20260812 16:57:46.913393 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:46.913996 8974 external_mini_cluster.cc:1428] Setting key 7a29394b13139b20f90af123938fdde9
I20260812 16:57:46.918455 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: Bootstrap starting.
I20260812 16:57:46.922734 19558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:46.923925 19558 log.cc:824] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:46.927304 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: No bootstrap required, opened a new log
Aug 12 16:57:46 dist-test-slave-z8x9 krb5kdc[19394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:46.935525 19558 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } }
I20260812 16:57:46.935967 19558 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:46.936108 19558 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c6d68b13a74544bcc0a4786845cc01, State: Initialized, Role: FOLLOWER
I20260812 16:57:46.936760 19558 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } }
I20260812 16:57:46.937042 19558 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:46.937229 19558 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:46.937438 19558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:46.940111 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:46.916054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53699 (local address 127.8.195.190:32957)
0812 16:57:46.916975 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:46.916988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:46.917033 (+ 45us) server_negotiation.cc:407] Connection header received
0812 16:57:46.919295 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:46.919319 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:46.919700 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:46.920017 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:46.922188 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:46.923673 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:46.925519 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:46.926595 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:46.926815 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:46.931601 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:46.931609 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:46.931629 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:46.931673 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:46.935072 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:46.935941 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:46.935949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:46.935962 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:46.936194 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:46.936685 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:46.936690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:46.936693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:46.937612 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:46.937990 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:46.938575 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 16:57:46.938931 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":442,"threads_started":1}
I20260812 16:57:46.940129 19558 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } }
I20260812 16:57:46.940764 19558 leader_election.cc:302] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5c6d68b13a74544bcc0a4786845cc01; no voters:
I20260812 16:57:46.941486 19558 leader_election.cc:288] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:46.941645 19568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:46.943753 19558 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:46.945331 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:46.948699 19568 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [term 1 LEADER]: Becoming Leader. State: Replica: c5c6d68b13a74544bcc0a4786845cc01, State: Running, Role: LEADER
I20260812 16:57:46.949384 19568 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } }
I20260812 16:57:46.954206 19570 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5c6d68b13a74544bcc0a4786845cc01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } } }
I20260812 16:57:46.954810 19570 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:46.954958 19574 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c6d68b13a74544bcc0a4786845cc01. Latest consensus state: current_term: 1 leader_uuid: "c5c6d68b13a74544bcc0a4786845cc01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c6d68b13a74544bcc0a4786845cc01" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 32957 } } }
I20260812 16:57:46.957680 19574 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01 [sys.catalog]: This master's current role is: LEADER
W20260812 16:57:46.960646 19593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:57:46.960866 19593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:57:46.961766 19600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:46.963891 19600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:46.973940 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:46.955853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44417 (local address 127.8.195.190:32957)
0812 16:57:46.956118 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:46.956133 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:46.956674 (+ 541us) server_negotiation.cc:407] Connection header received
0812 16:57:46.956943 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:46.956951 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:46.957082 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:46.957255 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:46.959538 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:46.960799 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:46.962801 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:46.963473 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:46.963651 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:46.967036 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:46.967042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:46.967047 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:46.967088 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:46.970426 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:46.971463 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:46.971469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:46.971473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:46.971653 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:46.972266 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:46.972271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:46.972275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:46.972651 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:46.972976 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:46.973394 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 16:57:46.973595 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260812 16:57:46.975059 19600 catalog_manager.cc:1347] Generated new cluster ID: c707fdcd4a264393be23804a82f21753
I20260812 16:57:46.975258 19600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:46.993335 19600 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:46.994661 19600 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:47.003680 19600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c6d68b13a74544bcc0a4786845cc01: Generated new TSK 0
I20260812 16:57:47.004580 19600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:47.015861 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19416
I20260812 16:57:47.017333 19544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:47.110538 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19416
W20260812 16:57:47.135308 19388 connection.cc:535] client connection to 127.8.195.190:32957 recv error: Network error: failed to read from TLS socket (remote: 127.8.195.190:32957): Connection reset by peer (error 104)
2026-08-12T16:57:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:47 dist-test-slave-z8x9 krb5kdc[19611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:47 dist-test-slave-z8x9 krb5kdc[19611](info): set up 2 sockets
Aug 12 16:57:47 dist-test-slave-z8x9 krb5kdc[19611](info): commencing operation
krb5kdc: starting...
W20260812 16:57:48.535583 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:48 dist-test-slave-z8x9 krb5kdc[19611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:48.880504 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35505
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:35505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:49.233647 19632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:49.233942 19632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:49.234127 19632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:49.241439 19632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:49.241561 19632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:49.241644 19632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:49.241751 19632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:49.241832 19632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:49.252033 19632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:35505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:35505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:49.253454 19632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:49.254769 19632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:49.266932 19646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:49.266979 19648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:49.267028 19645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:49.267911 19632 server_base.cc:1034] running on GCE node
I20260812 16:57:49.268657 19632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:49.270094 19632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:49.271297 19632 hybrid_clock.cc:648] HybridClock initialized: now 1786553869271235 us; error 51 us; skew 500 ppm
I20260812 16:57:49.271595 19632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:49 dist-test-slave-z8x9 krb5kdc[19611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553869, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:49.274828 19632 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:49.276258 19632 webserver.cc:460] Webserver started at http://127.8.195.190:42189/ using document root <none> and password file <none>
I20260812 16:57:49.276901 19632 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:49.277006 19632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:49.277351 19632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:49.280262 19632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "1ee341f4c5d04fcfaebb5ef54642a752"
format_stamp: "Formatted at 2026-08-12 16:57:49 on dist-test-slave-z8x9"
server_key: "33d71085fcda2fc2c5bb2e9d7ccd69ed"
server_key_iv: "6b9d74e3333dfaefc5a2bbf63329ea87"
server_key_version: "encryptionkey@0"
I20260812 16:57:49.280998 19632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "1ee341f4c5d04fcfaebb5ef54642a752"
format_stamp: "Formatted at 2026-08-12 16:57:49 on dist-test-slave-z8x9"
server_key: "33d71085fcda2fc2c5bb2e9d7ccd69ed"
server_key_iv: "6b9d74e3333dfaefc5a2bbf63329ea87"
server_key_version: "encryptionkey@0"
I20260812 16:57:49.286676 19632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:57:49.290107 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:49.291467 19632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:49.291608 19632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "1ee341f4c5d04fcfaebb5ef54642a752"
format_stamp: "Formatted at 2026-08-12 16:57:49 on dist-test-slave-z8x9"
server_key: "33d71085fcda2fc2c5bb2e9d7ccd69ed"
server_key_iv: "6b9d74e3333dfaefc5a2bbf63329ea87"
server_key_version: "encryptionkey@0"
I20260812 16:57:49.291769 19632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:49.310864 19632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:49.315038 19632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:49.315272 19632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:49.334070 19632 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:35505
I20260812 16:57:49.334110 19775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:35505 every 8 connection(s)
I20260812 16:57:49.335491 19632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:49.339196 19777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:49.345022 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19632
I20260812 16:57:49.345242 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:49.345840 8974 external_mini_cluster.cc:1428] Setting key 19fd3aafd6f005e8ef9104b756e743c7
I20260812 16:57:49.350219 19777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752: Bootstrap starting.
I20260812 16:57:49.353693 19777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:49.354755 19777 log.cc:824] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:49.358132 19777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752: No bootstrap required, opened a new log
I20260812 16:57:49.359896 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:49.347783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35875 (local address 127.8.195.190:35505)
0812 16:57:49.348681 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:49.348693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:49.348735 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:57:49.350545 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:49.350568 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:49.350952 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:49.351279 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:49.351944 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:49.353329 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:49.354836 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:49.355906 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:49.356118 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:49.357487 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:49.357494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:49.357509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:49.358042 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:49.358126 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:49.358151 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:49.358414 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:49.358701 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 16:57:49.358895 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":440,"threads_started":1}
I20260812 16:57:49.364099 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:49.366956 19777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } }
I20260812 16:57:49.367427 19777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:49.367563 19777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee341f4c5d04fcfaebb5ef54642a752, State: Initialized, Role: FOLLOWER
I20260812 16:57:49.368129 19777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } }
I20260812 16:57:49.368376 19777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:49.368543 19777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:49.368798 19777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:49.371944 19777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } }
I20260812 16:57:49.372539 19777 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee341f4c5d04fcfaebb5ef54642a752; no voters:
I20260812 16:57:49.373255 19777 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:49.373406 19794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:49.374459 19794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [term 1 LEADER]: Becoming Leader. State: Replica: 1ee341f4c5d04fcfaebb5ef54642a752, State: Running, Role: LEADER
I20260812 16:57:49.375136 19794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } }
I20260812 16:57:49.375948 19777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:49.376499 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:49.368821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53025 (local address 127.8.195.190:35505)
0812 16:57:49.369075 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:49.369084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:49.369602 (+ 518us) server_negotiation.cc:407] Connection header received
0812 16:57:49.369778 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:49.369782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:49.369849 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:49.369935 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:49.370537 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:49.371333 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:49.373274 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:49.374239 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:49.375076 (+ 837us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:49.375674 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:49.375678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:49.375681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:49.375697 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:49.375753 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:49.375768 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:49.375892 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:49.376090 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 16:57:49.376158 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 16:57:49.379719 19798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee341f4c5d04fcfaebb5ef54642a752. Latest consensus state: current_term: 1 leader_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } } }
I20260812 16:57:49.380029 19795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee341f4c5d04fcfaebb5ef54642a752" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 35505 } } }
I20260812 16:57:49.380338 19795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:49.380282 19798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:49.382406 19818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:49.387645 19818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:49.397457 19818 catalog_manager.cc:1347] Generated new cluster ID: 5d45cbac78d8468da031083493b6c430
I20260812 16:57:49.397603 19818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:49.415287 19818 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:49.417328 19818 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:49.428659 19818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee341f4c5d04fcfaebb5ef54642a752: Generated new TSK 0
I20260812 16:57:49.429579 19818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:49.444855 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19632
W20260812 16:57:49.446451 19769 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35875
W20260812 16:57:49.447414 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:49.447573 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19632
W20260812 16:57:49.476974 19609 connection.cc:535] client connection to 127.8.195.190:35505 recv error: Network error: failed to read from TLS socket (remote: 127.8.195.190:35505): Connection reset by peer (error 104)
2026-08-12T16:57:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:49 dist-test-slave-z8x9 krb5kdc[19849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:49 dist-test-slave-z8x9 krb5kdc[19849](info): set up 2 sockets
Aug 12 16:57:49 dist-test-slave-z8x9 krb5kdc[19849](info): commencing operation
krb5kdc: starting...
W20260812 16:57:50.877535 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.008s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:51 dist-test-slave-z8x9 krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:51.228953 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42357
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:51.577209 19874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:51.577504 19874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:51.577705 19874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:51.584878 19874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:51.584983 19874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:51.585112 19874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:51.585242 19874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:51.585323 19874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:51.595064 19874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:51.596473 19874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:51.597863 19874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:51.610482 19880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:51.610508 19882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:51.610437 19885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:51.611541 19874 server_base.cc:1034] running on GCE node
I20260812 16:57:51.612378 19874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:51.613901 19874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:51.615134 19874 hybrid_clock.cc:648] HybridClock initialized: now 1786553871615087 us; error 41 us; skew 500 ppm
I20260812 16:57:51.615500 19874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:51 dist-test-slave-z8x9 krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553871, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:51.619364 19874 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:51.620777 19874 webserver.cc:460] Webserver started at http://127.8.195.190:46873/ using document root <none> and password file <none>
I20260812 16:57:51.621539 19874 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:51.621647 19874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:51.621906 19874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:51.624825 19874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "f06fe40496ea489f849932cc9fcaf4a5"
format_stamp: "Formatted at 2026-08-12 16:57:51 on dist-test-slave-z8x9"
server_key: "5f05f0c1c253dcc6b273eece9ed01c05"
server_key_iv: "2e1f8d189fac66448760679334c71f92"
server_key_version: "encryptionkey@0"
I20260812 16:57:51.625758 19874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "f06fe40496ea489f849932cc9fcaf4a5"
format_stamp: "Formatted at 2026-08-12 16:57:51 on dist-test-slave-z8x9"
server_key: "5f05f0c1c253dcc6b273eece9ed01c05"
server_key_iv: "2e1f8d189fac66448760679334c71f92"
server_key_version: "encryptionkey@0"
I20260812 16:57:51.630924 19874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:57:51.634256 19896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:51.635537 19874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:57:51.635720 19874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "f06fe40496ea489f849932cc9fcaf4a5"
format_stamp: "Formatted at 2026-08-12 16:57:51 on dist-test-slave-z8x9"
server_key: "5f05f0c1c253dcc6b273eece9ed01c05"
server_key_iv: "2e1f8d189fac66448760679334c71f92"
server_key_version: "encryptionkey@0"
I20260812 16:57:51.635924 19874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:51.641885 19874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:51.646088 19874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:51.646416 19874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:51.666673 19874 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:42357
I20260812 16:57:51.666698 19998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:42357 every 8 connection(s)
I20260812 16:57:51.668227 19874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:51.672283 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 19874
I20260812 16:57:51.672462 20003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:51.672488 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:51.673054 8974 external_mini_cluster.cc:1428] Setting key 752fdaebe879f6ec9859c4e4b4fa362f
I20260812 16:57:51.681049 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:51.674973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53851 (local address 127.8.195.190:42357)
0812 16:57:51.675825 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:51.675840 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:51.675882 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:57:51.677665 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:51.677688 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:51.678039 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:51.678449 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:51.679479 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:51.679485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:51.679497 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:51.679513 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:51.679575 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:51.679595 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:51.679859 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:51.680018 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:57:51.680184 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":375,"threads_started":1}
I20260812 16:57:51.684610 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:51.686215 20003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5: Bootstrap starting.
I20260812 16:57:51.689687 20003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:51.691118 20003 log.cc:824] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:51.691999 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:51.689273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51869 (local address 127.8.195.190:42357)
0812 16:57:51.689464 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:51.689472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:51.689922 (+ 450us) server_negotiation.cc:407] Connection header received
0812 16:57:51.690080 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:51.690087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:51.690169 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:51.690307 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:51.691254 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:51.691260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:51.691265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:51.691283 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:51.691332 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:51.691354 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:51.691470 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:51.691553 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 16:57:51.691611 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 16:57:51.694414 20003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5: No bootstrap required, opened a new log
I20260812 16:57:51.702445 20003 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } }
I20260812 16:57:51.702812 20003 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:51.702888 20003 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06fe40496ea489f849932cc9fcaf4a5, State: Initialized, Role: FOLLOWER
I20260812 16:57:51.703387 20003 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } }
I20260812 16:57:51.703596 20003 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:51.703704 20003 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:51.703831 20003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:51.706115 20003 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } }
I20260812 16:57:51.706574 20003 leader_election.cc:302] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06fe40496ea489f849932cc9fcaf4a5; no voters:
I20260812 16:57:51.707195 20003 leader_election.cc:288] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:51.707322 20018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:51.708685 20018 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [term 1 LEADER]: Becoming Leader. State: Replica: f06fe40496ea489f849932cc9fcaf4a5, State: Running, Role: LEADER
I20260812 16:57:51.709342 20018 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } }
I20260812 16:57:51.709856 20003 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:51.712689 20024 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06fe40496ea489f849932cc9fcaf4a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } } }
I20260812 16:57:51.713201 20024 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:51.712678 20025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06fe40496ea489f849932cc9fcaf4a5. Latest consensus state: current_term: 1 leader_uuid: "f06fe40496ea489f849932cc9fcaf4a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06fe40496ea489f849932cc9fcaf4a5" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42357 } } }
I20260812 16:57:51.713354 20025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:51.717767 20031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:51.723273 20031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:51.732877 20031 catalog_manager.cc:1347] Generated new cluster ID: 92adbaf36efb461d8828f3a6f3a006c9
I20260812 16:57:51.733026 20031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:51.752924 20031 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:51.753492 20031 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:51.762817 20031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06fe40496ea489f849932cc9fcaf4a5: Generated new TSK 0
I20260812 16:57:51.763883 20031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:51.773221 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19874
W20260812 16:57:51.774336 19994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53851
W20260812 16:57:51.775080 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 19874: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:51.775245 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 19874
2026-08-12T16:57:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:51 dist-test-slave-z8x9 krb5kdc[20057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:51 dist-test-slave-z8x9 krb5kdc[20057](info): set up 2 sockets
Aug 12 16:57:51 dist-test-slave-z8x9 krb5kdc[20057](info): commencing operation
krb5kdc: starting...
W20260812 16:57:53.226473 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.405s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:53 dist-test-slave-z8x9 krb5kdc[20057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:53.584712 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:38021
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:38021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:53.936759 20078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:53.937117 20078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:53.937314 20078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:53.944608 20078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:53.944762 20078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:53.944890 20078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:53.945009 20078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:53.945089 20078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:53.955483 20078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:38021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:53.956940 20078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:53.958397 20078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:53.970752 20092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:53.970842 20089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:53.971040 20087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:53.971848 20078 server_base.cc:1034] running on GCE node
I20260812 16:57:53.972694 20078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:53.974316 20078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:53.975577 20078 hybrid_clock.cc:648] HybridClock initialized: now 1786553873975535 us; error 36 us; skew 500 ppm
I20260812 16:57:53.975947 20078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:53 dist-test-slave-z8x9 krb5kdc[20057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:53.984562 20078 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:53.986370 20078 webserver.cc:460] Webserver started at http://127.8.195.190:34637/ using document root <none> and password file <none>
I20260812 16:57:53.987090 20078 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:53.987244 20078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:53.987572 20078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:53.990722 20078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "8418c24b90cd4477bc96af6a38d6cb8f"
format_stamp: "Formatted at 2026-08-12 16:57:53 on dist-test-slave-z8x9"
server_key: "370042e71abb29a89d609a308f85e7ad"
server_key_iv: "29516bb1d2c27b466a5140ab7f3a4f35"
server_key_version: "encryptionkey@0"
I20260812 16:57:53.991515 20078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "8418c24b90cd4477bc96af6a38d6cb8f"
format_stamp: "Formatted at 2026-08-12 16:57:53 on dist-test-slave-z8x9"
server_key: "370042e71abb29a89d609a308f85e7ad"
server_key_iv: "29516bb1d2c27b466a5140ab7f3a4f35"
server_key_version: "encryptionkey@0"
I20260812 16:57:53.996789 20078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 16:57:54.000339 20107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:54.001730 20078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 16:57:54.001932 20078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "8418c24b90cd4477bc96af6a38d6cb8f"
format_stamp: "Formatted at 2026-08-12 16:57:53 on dist-test-slave-z8x9"
server_key: "370042e71abb29a89d609a308f85e7ad"
server_key_iv: "29516bb1d2c27b466a5140ab7f3a4f35"
server_key_version: "encryptionkey@0"
I20260812 16:57:54.002146 20078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:54.044876 20078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:54.049077 20078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:54.049417 20078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:54.068905 20078 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:38021
I20260812 16:57:54.068954 20224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:38021 every 8 connection(s)
I20260812 16:57:54.070400 20078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:54.074184 20225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:54.080726 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20078
I20260812 16:57:54.080928 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:54.081570 8974 external_mini_cluster.cc:1428] Setting key 1d2a68cd30910382b74ab01aa5afcd87
I20260812 16:57:54.085489 20225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f: Bootstrap starting.
I20260812 16:57:54.089141 20225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:54.090281 20225 log.cc:824] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:54.093371 20225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f: No bootstrap required, opened a new log
Aug 12 16:57:54 dist-test-slave-z8x9 krb5kdc[20057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:57:54.101843 20225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } }
I20260812 16:57:54.102237 20225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:54.102370 20225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8418c24b90cd4477bc96af6a38d6cb8f, State: Initialized, Role: FOLLOWER
I20260812 16:57:54.102941 20225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } }
I20260812 16:57:54.103211 20225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:54.103382 20225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:54.103579 20225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:54.105307 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:54.083433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46451 (local address 127.8.195.190:38021)
0812 16:57:54.084217 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:54.084229 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:54.084271 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:57:54.086083 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:54.086105 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:54.086469 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:54.086775 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:54.088683 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:54.090128 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:54.091728 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:54.092373 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:54.092497 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:54.097469 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:54.097477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:54.097491 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:54.097537 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:54.100806 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:54.101579 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:54.101586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:54.101592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:54.101755 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:54.102161 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:54.102167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:54.102172 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:54.102967 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:54.103292 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:54.103742 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 16:57:54.104185 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":371,"threads_started":1}
I20260812 16:57:54.106302 20225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } }
I20260812 16:57:54.106868 20225 leader_election.cc:302] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8418c24b90cd4477bc96af6a38d6cb8f; no voters:
I20260812 16:57:54.107596 20225 leader_election.cc:288] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:54.107717 20234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:54.109154 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:54.109261 20234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [term 1 LEADER]: Becoming Leader. State: Replica: 8418c24b90cd4477bc96af6a38d6cb8f, State: Running, Role: LEADER
I20260812 16:57:54.109812 20234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } }
I20260812 16:57:54.110087 20225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:54.112836 20235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } } }
I20260812 16:57:54.113354 20235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:54.118371 20238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8418c24b90cd4477bc96af6a38d6cb8f. Latest consensus state: current_term: 1 leader_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8418c24b90cd4477bc96af6a38d6cb8f" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38021 } } }
I20260812 16:57:54.120141 20238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:54.123960 20245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:54.130343 20245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:54.143723 20245 catalog_manager.cc:1347] Generated new cluster ID: 91f5616a7de74fac880317f3c5841ffd
I20260812 16:57:54.143847 20245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:54.144374 20230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:54.125721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59267 (local address 127.8.195.190:38021)
0812 16:57:54.125906 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:54.125914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:54.125983 (+ 69us) server_negotiation.cc:407] Connection header received
0812 16:57:54.126255 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:54.126262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:54.126372 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:54.126513 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:54.128560 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:54.129523 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:54.131733 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:54.132758 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:54.132889 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:54.137378 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:54.137389 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:54.137396 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:57:54.137458 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:54.141033 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:54.142007 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:54.142013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:54.142016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:54.142174 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:57:54.142659 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:57:54.142664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:57:54.142668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:57:54.143125 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:54.143375 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:54.143937 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 16:57:54.144119 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:57:54.162405 20245 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:54.164417 20245 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:54.177773 20245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8418c24b90cd4477bc96af6a38d6cb8f: Generated new TSK 0
I20260812 16:57:54.178822 20245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:54.186501 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20078
I20260812 16:57:54.187639 20216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:57:54.277928 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20078
2026-08-12T16:57:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:54 dist-test-slave-z8x9 krb5kdc[20287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:54 dist-test-slave-z8x9 krb5kdc[20287](info): set up 2 sockets
Aug 12 16:57:54 dist-test-slave-z8x9 krb5kdc[20287](info): commencing operation
krb5kdc: starting...
W20260812 16:57:55.746816 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.419s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:55 dist-test-slave-z8x9 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:56.096459 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:38277
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:38277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:56.477401 20313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:56.477706 20313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:56.477802 20313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:56.485236 20313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:56.485370 20313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:56.485419 20313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:56.485482 20313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:56.485529 20313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:56.495626 20313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:38277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:38277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:56.496913 20313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:56.498245 20313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:56.510443 20331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:56.510517 20328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:56.510496 20324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:56.511793 20313 server_base.cc:1034] running on GCE node
I20260812 16:57:56.512708 20313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:56.514344 20313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:56.515594 20313 hybrid_clock.cc:648] HybridClock initialized: now 1786553876515552 us; error 37 us; skew 500 ppm
I20260812 16:57:56.515965 20313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:56 dist-test-slave-z8x9 krb5kdc[20287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553876, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:56.519593 20313 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:56.521255 20313 webserver.cc:460] Webserver started at http://127.8.195.190:40879/ using document root <none> and password file <none>
I20260812 16:57:56.521966 20313 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:56.522130 20313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:56.522472 20313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:56.525700 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "f6d3670d87fa4a44b5e044dcbde0a677"
format_stamp: "Formatted at 2026-08-12 16:57:56 on dist-test-slave-z8x9"
server_key: "355649c1e91a3ff37d696c935814dd73"
server_key_iv: "5062869c2d348dad4dcafc5b6e4287d7"
server_key_version: "encryptionkey@0"
I20260812 16:57:56.526561 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "f6d3670d87fa4a44b5e044dcbde0a677"
format_stamp: "Formatted at 2026-08-12 16:57:56 on dist-test-slave-z8x9"
server_key: "355649c1e91a3ff37d696c935814dd73"
server_key_iv: "5062869c2d348dad4dcafc5b6e4287d7"
server_key_version: "encryptionkey@0"
I20260812 16:57:56.532262 20313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 16:57:56.536001 20346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:56.537447 20313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 16:57:56.537644 20313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "f6d3670d87fa4a44b5e044dcbde0a677"
format_stamp: "Formatted at 2026-08-12 16:57:56 on dist-test-slave-z8x9"
server_key: "355649c1e91a3ff37d696c935814dd73"
server_key_iv: "5062869c2d348dad4dcafc5b6e4287d7"
server_key_version: "encryptionkey@0"
I20260812 16:57:56.537863 20313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:56.578966 20313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:56.583535 20313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:56.583877 20313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:56.602419 20313 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:38277
I20260812 16:57:56.602427 20450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:38277 every 8 connection(s)
I20260812 16:57:56.603812 20313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:56.604318 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20313
I20260812 16:57:56.604521 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:56.605239 8974 external_mini_cluster.cc:1428] Setting key 1f7c63ebc33015d9574346b9723ef759
I20260812 16:57:56.608206 20452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:56.618610 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:56.607164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51491 (local address 127.8.195.190:38277)
0812 16:57:56.607997 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:56.608008 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:56.608049 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:57:56.609831 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:56.609852 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:56.610209 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:56.610517 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:56.611124 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:56.612430 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:56.614273 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:56.614909 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:56.615034 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:56.616352 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:56.616359 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:56.616373 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:56.616892 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:56.616965 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:56.616988 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:56.617078 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:56.617372 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 16:57:56.617636 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":388,"threads_started":1}
I20260812 16:57:56.620229 20452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677: Bootstrap starting.
I20260812 16:57:56.622205 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:56.623442 20452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:56.624429 20452 log.cc:824] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:56.627960 20452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677: No bootstrap required, opened a new log
I20260812 16:57:56.633668 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:56.626709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48231 (local address 127.8.195.190:38277)
0812 16:57:56.626910 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:56.626919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:56.627432 (+ 513us) server_negotiation.cc:407] Connection header received
0812 16:57:56.627645 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:56.627649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:56.627707 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:56.627820 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:56.628405 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:56.629131 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:56.631093 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:57:56.631551 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:57:56.631651 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:57:56.632839 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:56.632844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:56.632847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:56.632864 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:56.632903 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:56.632917 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:56.632993 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:56.633154 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 16:57:56.633294 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 16:57:56.637318 20452 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } }
I20260812 16:57:56.637714 20452 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:56.637832 20452 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d3670d87fa4a44b5e044dcbde0a677, State: Initialized, Role: FOLLOWER
I20260812 16:57:56.638581 20452 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } }
I20260812 16:57:56.638906 20452 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:56.639073 20452 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:56.639285 20452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:56.641613 20452 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } }
I20260812 16:57:56.642125 20452 leader_election.cc:302] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6d3670d87fa4a44b5e044dcbde0a677; no voters:
I20260812 16:57:56.642746 20452 leader_election.cc:288] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:56.642925 20467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:56.643676 20467 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [term 1 LEADER]: Becoming Leader. State: Replica: f6d3670d87fa4a44b5e044dcbde0a677, State: Running, Role: LEADER
I20260812 16:57:56.644213 20467 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } }
I20260812 16:57:56.644925 20452 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:56.647549 20469 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6d3670d87fa4a44b5e044dcbde0a677. Latest consensus state: current_term: 1 leader_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } } }
I20260812 16:57:56.647606 20468 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6d3670d87fa4a44b5e044dcbde0a677" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 38277 } } }
I20260812 16:57:56.647993 20469 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:56.648018 20468 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677 [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:56.650148 20483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:56.654920 20483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:56.666322 20483 catalog_manager.cc:1347] Generated new cluster ID: cd366ab95ab5479382d453d4ade2c35a
I20260812 16:57:56.666486 20483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:56.697793 20483 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:56.699123 20483 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:56.708601 20483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6d3670d87fa4a44b5e044dcbde0a677: Generated new TSK 0
I20260812 16:57:56.709468 20483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:56.723794 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20313
W20260812 16:57:56.724851 20444 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51491
W20260812 16:57:56.725682 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20313: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:56.725836 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20313
2026-08-12T16:57:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:56 dist-test-slave-z8x9 krb5kdc[20509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:56 dist-test-slave-z8x9 krb5kdc[20509](info): set up 2 sockets
Aug 12 16:57:56 dist-test-slave-z8x9 krb5kdc[20509](info): commencing operation
krb5kdc: starting...
W20260812 16:57:58.157135 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.004s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:57:58 dist-test-slave-z8x9 krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:57:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:57:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:57:58.521775 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36887
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:57:58.883787 20531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:57:58.884140 20531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:57:58.884459 20531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:57:58.891968 20531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:57:58.892143 20531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:57:58.892302 20531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:57:58.892429 20531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:57:58.892518 20531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:57:58.902758 20531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:57:58.904175 20531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:57:58.905618 20531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:57:58.918037 20546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:58.918074 20551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:57:58.917995 20547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:57:58.918740 20531 server_base.cc:1034] running on GCE node
I20260812 16:57:58.919586 20531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:57:58.921060 20531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:57:58.922343 20531 hybrid_clock.cc:648] HybridClock initialized: now 1786553878922287 us; error 47 us; skew 500 ppm
I20260812 16:57:58.922729 20531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:57:58 dist-test-slave-z8x9 krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553878, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:57:58.926054 20531 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:57:58.927556 20531 webserver.cc:460] Webserver started at http://127.8.195.190:41235/ using document root <none> and password file <none>
I20260812 16:57:58.928267 20531 fs_manager.cc:359] Metadata directory not provided
I20260812 16:57:58.928416 20531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:57:58.928736 20531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:57:58.931770 20531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "96b4db408ed74959a2eb4dba4238cf2c"
format_stamp: "Formatted at 2026-08-12 16:57:58 on dist-test-slave-z8x9"
server_key: "8e6e80c42b32db969f488fd0ff633e71"
server_key_iv: "4fc5d12d9a68e06905c93ab258ff2b0b"
server_key_version: "encryptionkey@0"
I20260812 16:57:58.932562 20531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "96b4db408ed74959a2eb4dba4238cf2c"
format_stamp: "Formatted at 2026-08-12 16:57:58 on dist-test-slave-z8x9"
server_key: "8e6e80c42b32db969f488fd0ff633e71"
server_key_iv: "4fc5d12d9a68e06905c93ab258ff2b0b"
server_key_version: "encryptionkey@0"
I20260812 16:57:58.937803 20531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:57:58.941423 20565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:57:58.942817 20531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:57:58.943042 20531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "96b4db408ed74959a2eb4dba4238cf2c"
format_stamp: "Formatted at 2026-08-12 16:57:58 on dist-test-slave-z8x9"
server_key: "8e6e80c42b32db969f488fd0ff633e71"
server_key_iv: "4fc5d12d9a68e06905c93ab258ff2b0b"
server_key_version: "encryptionkey@0"
I20260812 16:57:58.943270 20531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:57:58.949338 20531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:57:58.954293 20531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:57:58.954644 20531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:57:58.974814 20531 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:36887
I20260812 16:57:58.974850 20666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:36887 every 8 connection(s)
I20260812 16:57:58.976480 20531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:57:58.976928 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20531
I20260812 16:57:58.977120 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:57:58.977800 8974 external_mini_cluster.cc:1428] Setting key a444aaee0118f1bcb562a5fad549145b
I20260812 16:57:58.981257 20668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:57:58.991153 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:58.979797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40001 (local address 127.8.195.190:36887)
0812 16:57:58.980678 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:58.980690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:58.980732 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:57:58.982534 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:58.982557 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:58.982891 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:58.983218 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:58.989411 (+ 6193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:58.989417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:58.989432 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:58.989451 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:58.989542 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:58.989568 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:58.989759 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:58.989898 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 16:57:58.990116 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":417,"threads_started":1}
I20260812 16:57:58.993314 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c: Bootstrap starting.
I20260812 16:57:58.995178 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:57:58.996832 20668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c: Neither blocks nor log segments found. Creating new log.
I20260812 16:57:58.998111 20668 log.cc:824] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c: Log is configured to *not* fsync() on all Append() calls
I20260812 16:57:59.002118 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c: No bootstrap required, opened a new log
I20260812 16:57:59.002977 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:57:58.999950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54821 (local address 127.8.195.190:36887)
0812 16:57:59.000202 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:57:59.000212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:57:59.000626 (+ 414us) server_negotiation.cc:407] Connection header received
0812 16:57:59.000831 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:57:59.000835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:57:59.000883 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:57:59.000969 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:57:59.002232 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:57:59.002240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:57:59.002253 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 16:57:59.002300 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:57:59.002342 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 16:57:59.002360 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:57:59.002455 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 16:57:59.002605 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 16:57:59.002659 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 16:57:59.010151 20668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } }
I20260812 16:57:59.010589 20668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 16:57:59.010726 20668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b4db408ed74959a2eb4dba4238cf2c, State: Initialized, Role: FOLLOWER
I20260812 16:57:59.011353 20668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } }
I20260812 16:57:59.011596 20668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 16:57:59.011770 20668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 16:57:59.011958 20668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:57:59.014370 20668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } }
I20260812 16:57:59.014916 20668 leader_election.cc:302] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b4db408ed74959a2eb4dba4238cf2c; no voters:
I20260812 16:57:59.015612 20668 leader_election.cc:288] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 16:57:59.015796 20688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:57:59.016499 20688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [term 1 LEADER]: Becoming Leader. State: Replica: 96b4db408ed74959a2eb4dba4238cf2c, State: Running, Role: LEADER
I20260812 16:57:59.017077 20688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } }
I20260812 16:57:59.017930 20668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:57:59.020406 20690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b4db408ed74959a2eb4dba4238cf2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } } }
I20260812 16:57:59.020397 20691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96b4db408ed74959a2eb4dba4238cf2c. Latest consensus state: current_term: 1 leader_uuid: "96b4db408ed74959a2eb4dba4238cf2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b4db408ed74959a2eb4dba4238cf2c" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36887 } } }
I20260812 16:57:59.020897 20691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:59.020895 20690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c [sys.catalog]: This master's current role is: LEADER
I20260812 16:57:59.023831 20705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:57:59.031006 20705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:57:59.045862 20705 catalog_manager.cc:1347] Generated new cluster ID: fda6106ef8a3443cad7e9ae444a80d46
I20260812 16:57:59.046000 20705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:57:59.057713 20705 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:57:59.058298 20705 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:57:59.071461 20705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96b4db408ed74959a2eb4dba4238cf2c: Generated new TSK 0
I20260812 16:57:59.072628 20705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:57:59.085093 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20531
W20260812 16:57:59.086356 20659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40001
W20260812 16:57:59.087097 8974 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 16:57:59.087280 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20531
2026-08-12T16:57:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2361 ms)
[----------] 18 tests from AuthTokenIssuingTest (43311 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:57:59 dist-test-slave-z8x9 krb5kdc[20736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:57:59 dist-test-slave-z8x9 krb5kdc[20736](info): set up 2 sockets
Aug 12 16:57:59 dist-test-slave-z8x9 krb5kdc[20736](info): commencing operation
krb5kdc: starting...
W20260812 16:58:00.510721 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.372s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:58:00 dist-test-slave-z8x9 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:58:00.876900 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36833
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:58:01.231772 20769 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:01.232172 20769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:01.232309 20769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:01.232445 20769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:01.239861 20769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:01.239959 20769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:01.240063 20769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:01.240140 20769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:01.240281 20769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:01.250489 20769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:36833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:01.251860 20769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:01.253229 20769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:01.265893 20785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:01.265944 20789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:01.265985 20781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:01.266948 20769 server_base.cc:1034] running on GCE node
I20260812 16:58:01.267768 20769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:01.269150 20769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:01.270457 20769 hybrid_clock.cc:648] HybridClock initialized: now 1786553881270394 us; error 34 us; skew 500 ppm
I20260812 16:58:01.270800 20769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:01 dist-test-slave-z8x9 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:01.274088 20769 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:58:01.275677 20769 webserver.cc:460] Webserver started at http://127.8.195.190:37971/ using document root <none> and password file <none>
I20260812 16:58:01.276382 20769 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:01.276527 20769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:01.276861 20769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:01.280164 20769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "bb5ee8e9f2274a55bfda61fdf121800b"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "762fd07df7dce263944466d3d3984ec1"
server_key_iv: "b33fb0db6fab098fc9864f0db2768716"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.280995 20769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "bb5ee8e9f2274a55bfda61fdf121800b"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "762fd07df7dce263944466d3d3984ec1"
server_key_iv: "b33fb0db6fab098fc9864f0db2768716"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.286537 20769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 16:58:01.290020 20806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:01.291378 20769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:58:01.291584 20769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "bb5ee8e9f2274a55bfda61fdf121800b"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "762fd07df7dce263944466d3d3984ec1"
server_key_iv: "b33fb0db6fab098fc9864f0db2768716"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.291824 20769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:01.307838 20769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:01.312435 20769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:01.312755 20769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:01.332629 20769 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:36833
I20260812 16:58:01.332710 20924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:36833 every 8 connection(s)
I20260812 16:58:01.334152 20769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:58:01.338370 20927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:01.342514 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20769
I20260812 16:58:01.342813 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:58:01.343756 8974 external_mini_cluster.cc:1428] Setting key 5c05fa57ddf6c849be6e4cf9f9b264eb
I20260812 16:58:01.348074 20927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:01 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:01.376768 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.352482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36833 (local address 127.0.0.1:34214)
0812 16:58:01.353543 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.353609 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:01.356940 (+ 3331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:01.358158 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:01.358169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:01.359153 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:01.359749 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.359780 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.361479 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.361489 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.363282 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.363330 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.364770 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.364793 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.364934 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.365733 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:01.365756 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:01.368009 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:01.372658 (+ 4649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.372669 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.372680 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.373099 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.373607 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.373610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.373613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.373798 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.374860 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:01.374866 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:01.375063 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 16:58:01.375277 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 16:58:01.375704 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":416,"threads_started":1}
I20260812 16:58:01.376852 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.352482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.8.195.190:36833)
0812 16:58:01.354703 (+ 2221us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:01.354709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:01.354753 (+ 44us) server_negotiation.cc:407] Connection header received
0812 16:58:01.357491 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:01.357502 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:01.357871 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:01.358119 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:01.359999 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.361300 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.363480 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.364512 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.364721 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.368271 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:01.368285 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:01.368305 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:01.368348 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:01.372514 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.373258 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.373270 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.373276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.373480 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.373928 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.373937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.373943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.374706 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:01.374925 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:01.375467 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 16:58:01.375958 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":2027,"spinlock_wait_cycles":17152,"thread_start_us":1653,"threads_started":1}
I20260812 16:58:01.380144 20927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:01.381626 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.381021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39832)
0812 16:58:01.381376 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.381505 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
W20260812 16:58:01.381928 20815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.195.189@KRBTEST.COM; defaulting to no policy
I20260812 16:58:01.383591 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.383201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39844)
0812 16:58:01.383382 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.383464 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:58:01.383817 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:34925: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
I20260812 16:58:01.425727 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:34925
--webserver_interface=127.8.195.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:58:01.433198 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } attempt: 1
I20260812 16:58:01.434432 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.433954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39846)
0812 16:58:01.434157 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.434291 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 16:58:01.435501 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.435195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39850)
0812 16:58:01.435319 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.435384 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 16:58:01.435779 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:34925: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
I20260812 16:58:01.504065 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } attempt: 2
I20260812 16:58:01.505419 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.504817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39856)
0812 16:58:01.505095 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.505233 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:58:01.506595 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.506094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39862)
0812 16:58:01.506273 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.506392 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 16:58:01.506841 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:34925: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
I20260812 16:58:01.598146 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } attempt: 3
I20260812 16:58:01.599545 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.598932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39864)
0812 16:58:01.599202 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.599341 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 16:58:01.600824 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.600320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39878)
0812 16:58:01.600508 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.600614 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 16:58:01.601048 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:34925: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
I20260812 16:58:01.744374 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } attempt: 4
I20260812 16:58:01.745780 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.745079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39892)
0812 16:58:01.745438 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.745579 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260812 16:58:01.746925 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.746491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39906)
0812 16:58:01.746667 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.746778 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 16:58:01.747195 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:34925: Network error: Client connection negotiation failed: client connection to 127.8.195.189:34925: connect: Connection refused (error 111)
W20260812 16:58:01.775940 20942 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:01.776310 20942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:01.776437 20942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:01.776587 20942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:01.784062 20942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:01.784157 20942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:01.784255 20942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:01.784356 20942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:01.784472 20942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:01.794349 20942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:34925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.195.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:01.795887 20942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:01.797403 20942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:01.809600 20956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:01.809702 20952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:01.809648 20951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:01.810491 20942 server_base.cc:1034] running on GCE node
I20260812 16:58:01.811316 20942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:01.812795 20942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:01.814041 20942 hybrid_clock.cc:648] HybridClock initialized: now 1786553881813978 us; error 38 us; skew 500 ppm
I20260812 16:58:01.814404 20942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:01 dist-test-slave-z8x9 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:01.817636 20942 init.cc:377] Logged in from keytab as kudu/127.8.195.189@KRBTEST.COM (short username kudu)
I20260812 16:58:01.819177 20942 webserver.cc:460] Webserver started at http://127.8.195.189:36503/ using document root <none> and password file <none>
I20260812 16:58:01.819881 20942 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:01.820020 20942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:01.820358 20942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:01.823446 20942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data/instance:
uuid: "86db490bc8ef4eb1ae7369d798c5fe99"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "7d9a6b0011e4b3eed5e58ad3b3175f7f"
server_key_iv: "0f1b1a506cf5c2800d1da76a47c97a08"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.824250 20942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal/instance:
uuid: "86db490bc8ef4eb1ae7369d798c5fe99"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "7d9a6b0011e4b3eed5e58ad3b3175f7f"
server_key_iv: "0f1b1a506cf5c2800d1da76a47c97a08"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.829671 20942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 16:58:01.833078 20975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:01.834446 20942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:58:01.834632 20942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "86db490bc8ef4eb1ae7369d798c5fe99"
format_stamp: "Formatted at 2026-08-12 16:58:01 on dist-test-slave-z8x9"
server_key: "7d9a6b0011e4b3eed5e58ad3b3175f7f"
server_key_iv: "0f1b1a506cf5c2800d1da76a47c97a08"
server_key_version: "encryptionkey@0"
I20260812 16:58:01.834851 20942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:01.858163 20942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:01.862807 20942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:01.863040 20942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:01.881151 20942 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.189:34925
I20260812 16:58:01.881160 21099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.189:34925 every 8 connection(s)
I20260812 16:58:01.882557 20942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
I20260812 16:58:01.886308 21101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:01.889834 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 20942
I20260812 16:58:01.890040 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-1/wal/instance
I20260812 16:58:01.890504 8974 external_mini_cluster.cc:1428] Setting key 57b0412a3bce99c4ffcfa0f9993d7555
I20260812 16:58:01.893006 21101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:01 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:01.917212 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.896355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.8.195.190:36833)
0812 16:58:01.896913 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:01.896918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:01.898047 (+ 1129us) server_negotiation.cc:407] Connection header received
0812 16:58:01.900129 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:01.900135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:01.900235 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:01.900364 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:01.902099 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.902877 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.904540 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.905220 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.905384 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.909155 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:01.909166 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:01.909217 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:01.909282 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:01.913562 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.914337 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.914346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.914353 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.914522 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.914936 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.914945 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.914952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.915406 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:01.915577 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:01.916296 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 16:58:01.916592 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":300,"threads_started":1}
I20260812 16:58:01.917783 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.897021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36833 (local address 127.0.0.1:34216)
0812 16:58:01.897861 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.897913 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:01.899838 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:01.900624 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:01.900633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:01.901273 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:01.901859 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.901877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.903063 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.903067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.904383 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.904396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.905484 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.905492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.905843 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.906694 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:01.906722 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:01.908904 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:01.913734 (+ 4830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.913741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.913754 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.914200 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.914642 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.914646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.914648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.914820 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.915542 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:01.915549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:01.915906 (+ 357us) client_negotiation.cc:769] Sending connection context
0812 16:58:01.916140 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 16:58:01.916553 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":324,"threads_started":1}
I20260812 16:58:01.919763 21101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.195.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.188@KRBTEST.COM" created.
Aug 12 16:58:01 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:01.940371 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.920744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39912)
0812 16:58:01.920927 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.920950 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:01.921063 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:01.922505 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:01.922509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:01.922886 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:01.923029 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.923036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.925509 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.925513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.926393 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:01.926404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.927772 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.927776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:01.927990 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.928730 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:01.928754 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:01.932448 (+ 3694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:01.937062 (+ 4614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.937070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.937077 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.937629 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.938066 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:01.938073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:01.938078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:01.938268 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:01.939148 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:01.939155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:01.939318 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 16:58:01.939454 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 16:58:01.939893 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":46}
I20260812 16:58:01.940587 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.921059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.8.195.189:34925)
0812 16:58:01.921925 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:01.921937 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:01.921974 (+ 37us) server_negotiation.cc:407] Connection header received
0812 16:58:01.922038 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:01.922050 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:01.922318 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:01.922498 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:01.923215 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.925352 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.926547 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:01.927607 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:01.927843 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:01.932704 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:01.932722 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:01.932744 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:01.932806 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:01.936878 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.937771 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.937783 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.937796 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.937953 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:01.938373 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:01.938380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:01.938386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:01.938966 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:01.939188 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:01.939651 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 16:58:01.940068 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":625,"spinlock_wait_cycles":28032,"thread_start_us":453,"threads_started":1}
I20260812 16:58:01.943596 21101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:01.944919 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.944311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49914)
0812 16:58:01.944544 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.944701 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 16:58:01.945253 20978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:01.946944 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.946506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49916)
0812 16:58:01.946678 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.946783 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:58:01.947203 21101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
I20260812 16:58:01.972860 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:37643
--webserver_interface=127.8.195.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:58:01.996569 21101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 1
I20260812 16:58:01.997980 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.997480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49922)
0812 16:58:01.997680 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.997806 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 16:58:01.999068 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:01.998685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49938)
0812 16:58:01.998877 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:01.998957 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 16:58:01.999346 21101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.046494 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } attempt: 5
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:02.061653 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.047291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:39928)
0812 16:58:02.047529 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.047552 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.047689 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.048113 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.048116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.048523 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.048701 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.048709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.050151 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.050154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.051012 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.051023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.052106 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.052110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.052301 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.052963 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.052980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.055091 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.059122 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.059126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.059129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.059521 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.059918 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.059921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.059923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.060051 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.060639 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.060643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.060840 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.061014 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.061378 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 16:58:02.061704 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.047533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.8.195.189:34925)
0812 16:58:02.047781 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.047788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.047812 (+ 24us) server_negotiation.cc:407] Connection header received
0812 16:58:02.047880 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.047885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.047967 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.048081 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.048875 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.050015 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.051176 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.051966 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.052106 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.055297 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.055304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.055308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.055347 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.058987 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.059639 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.059644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.059647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.059811 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.060153 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.060158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.060161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.060526 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.060703 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.061152 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.061404 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 16:58:02.062904 20927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:02.063874 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.063510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49952)
0812 16:58:02.063666 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.063739 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 16:58:02.064942 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.064478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49960)
0812 16:58:02.064659 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.064791 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:58:02.065317 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.067564 21101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 2
I20260812 16:58:02.068445 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.068085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49974)
0812 16:58:02.068236 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.068318 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 16:58:02.069396 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.069025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49978)
0812 16:58:02.069203 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.069288 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 16:58:02.069628 21101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.116576 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 1
I20260812 16:58:02.118098 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.117506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49988)
0812 16:58:02.117760 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.117882 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 16:58:02.119280 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.118882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:49996)
0812 16:58:02.119050 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.119144 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 16:58:02.119537 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.160897 21101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 3
I20260812 16:58:02.162127 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.161636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50004)
0812 16:58:02.161845 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.161946 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 16:58:02.163301 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.162803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50018)
0812 16:58:02.163040 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.163168 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260812 16:58:02.163542 21101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.187805 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 2
I20260812 16:58:02.189347 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.188594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50022)
0812 16:58:02.188902 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.189070 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 16:58:02.190625 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.190165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50034)
0812 16:58:02.190360 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.190470 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 16:58:02.190902 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.297196 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 3
I20260812 16:58:02.298417 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.297841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50042)
0812 16:58:02.298069 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.298185 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 16:58:02.299592 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.299112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50048)
0812 16:58:02.299353 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.299453 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260812 16:58:02.299847 20927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
I20260812 16:58:02.306766 21101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 4
I20260812 16:58:02.307855 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.307396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50060)
0812 16:58:02.307615 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.307697 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 16:58:02.308828 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.308503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:50062)
0812 16:58:02.308644 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.308718 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 16:58:02.309041 21101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:37643: Network error: Client connection negotiation failed: client connection to 127.8.195.188:37643: connect: Connection refused (error 111)
W20260812 16:58:02.332212 21116 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:02.332624 21116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:02.332759 21116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:02.332903 21116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:02.340842 21116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:02.340971 21116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:02.341101 21116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:02.341212 21116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:02.341311 21116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:02.351418 21116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:36833,127.8.195.189:34925,127.8.195.188:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:37643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.195.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:02.352777 21116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:02.354223 21116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:02.366632 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:02.366632 21126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:02.366824 21134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:02.367683 21116 server_base.cc:1034] running on GCE node
I20260812 16:58:02.368614 21116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:02.370105 21116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:02.371376 21116 hybrid_clock.cc:648] HybridClock initialized: now 1786553882371311 us; error 39 us; skew 500 ppm
I20260812 16:58:02.371732 21116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:02.375108 21116 init.cc:377] Logged in from keytab as kudu/127.8.195.188@KRBTEST.COM (short username kudu)
I20260812 16:58:02.376669 21116 webserver.cc:460] Webserver started at http://127.8.195.188:36319/ using document root <none> and password file <none>
I20260812 16:58:02.377518 21116 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:02.377660 21116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:02.377990 21116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:02.380950 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data/instance:
uuid: "235d3aa566054f6bb878d246fb02827e"
format_stamp: "Formatted at 2026-08-12 16:58:02 on dist-test-slave-z8x9"
server_key: "a01c067c4ead8c4913f545575497986c"
server_key_iv: "144a4a3931f25f8474a40c6a5809d3da"
server_key_version: "encryptionkey@0"
I20260812 16:58:02.381878 21116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal/instance:
uuid: "235d3aa566054f6bb878d246fb02827e"
format_stamp: "Formatted at 2026-08-12 16:58:02 on dist-test-slave-z8x9"
server_key: "a01c067c4ead8c4913f545575497986c"
server_key_iv: "144a4a3931f25f8474a40c6a5809d3da"
server_key_version: "encryptionkey@0"
I20260812 16:58:02.387080 21116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 16:58:02.390532 21146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:02.391889 21116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:58:02.392042 21116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "235d3aa566054f6bb878d246fb02827e"
format_stamp: "Formatted at 2026-08-12 16:58:02 on dist-test-slave-z8x9"
server_key: "a01c067c4ead8c4913f545575497986c"
server_key_iv: "144a4a3931f25f8474a40c6a5809d3da"
server_key_version: "encryptionkey@0"
I20260812 16:58:02.392233 21116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:02.407338 21116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:02.411440 21116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:02.411677 21116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:02.430701 21116 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.188:37643
I20260812 16:58:02.430708 21250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.188:37643 every 8 connection(s)
I20260812 16:58:02.432075 21116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
I20260812 16:58:02.435743 21251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:02.438359 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21116
I20260812 16:58:02.438607 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786553491558846-8974-0/minicluster-data/master-2/wal/instance
I20260812 16:58:02.439201 8974 external_mini_cluster.cc:1428] Setting key 8a362c566487a66339df6f7d7ebdb246
I20260812 16:58:02.443743 21251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:02.464694 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.446661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.8.195.190:36833)
0812 16:58:02.447191 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.447198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.447708 (+ 510us) server_negotiation.cc:407] Connection header received
0812 16:58:02.447910 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.447914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.447978 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.448070 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.449504 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.450528 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.452441 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.453118 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.453373 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.457393 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.457402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.457407 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.457460 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.461018 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.462013 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.462019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.462022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.462203 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.462761 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.462767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.462770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.463150 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.463335 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.463972 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.464241 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":246,"threads_started":1}
I20260812 16:58:02.465765 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.441381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.8.195.188:37643)
0812 16:58:02.442224 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.442236 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.442278 (+ 42us) server_negotiation.cc:407] Connection header received
0812 16:58:02.443887 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.443909 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.444291 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.444632 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.446535 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.448725 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.450477 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.451572 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.451758 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.456986 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.456996 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.457008 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.457039 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.460989 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.462040 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.462049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.462052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.462250 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.462749 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.462754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.462757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.463494 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.463740 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.464161 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.464554 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":619,"thread_start_us":402,"threads_started":1}
I20260812 16:58:02.465863 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.446724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:36833 (local address 127.0.0.1:45642)
0812 16:58:02.447538 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.447600 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.447769 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.448158 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.448164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.449018 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.449351 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.449360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.450675 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.450680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.452249 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.452265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.453403 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.453408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.453539 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.454171 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.454195 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.457113 (+ 2918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.461200 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.461214 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.461227 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.461864 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.462370 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.462375 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.462378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.462632 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.463285 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.463292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.463497 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.463826 (+ 329us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.464395 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":394,"threads_started":1}
I20260812 16:58:02.467625 21251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:02.477069 20927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 4
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:02.485275 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.468649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:34925 (local address 127.0.0.1:34588)
0812 16:58:02.468943 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.468965 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.469073 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.469865 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.469872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.470413 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.470639 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.470655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.471998 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.472005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.473056 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.473069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.473926 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.473932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.474135 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.474817 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.474836 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.477601 (+ 2765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.482174 (+ 4573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.482180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.482185 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.482801 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.483262 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.483267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.483272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.483451 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.484180 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.484185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.484378 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.484538 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.484881 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260812 16:58:02.486279 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.468814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.8.195.189:34925)
0812 16:58:02.469092 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.469099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.469145 (+ 46us) server_negotiation.cc:407] Connection header received
0812 16:58:02.469256 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.469305 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.469446 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.469603 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.470900 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.471828 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.473272 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.473756 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.473916 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.477862 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.477870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.477876 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.477920 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.482005 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.482941 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.482949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.482954 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.483117 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.483578 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.483584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.483590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.484008 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.484221 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.485634 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.485879 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:02.487022 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.469657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45656 (local address 127.8.195.190:36833)
0812 16:58:02.470398 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.470405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.470426 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:58:02.470493 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.470498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.470614 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.470719 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.472464 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.473243 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.474444 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.474891 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.475034 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.480141 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.480146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.480149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.480178 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.483550 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.484351 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.484360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.484366 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.484554 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.485033 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.485038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.485042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.485699 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.486094 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.486392 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.486656 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607}
I20260812 16:58:02.487977 21251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:02.494952 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.477852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:52676)
0812 16:58:02.478040 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.478062 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.478169 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.478610 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.478616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.479174 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.479346 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.479355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.480758 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.480767 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.482640 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.482661 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.483719 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.483726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.483931 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.484649 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.484675 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.488202 (+ 3527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.492372 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.492377 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.492382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.492770 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.493216 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.493222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.493227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.493419 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.494151 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.494157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.494322 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.494421 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.494625 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 16:58:02.495867 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.478143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.8.195.188:37643)
0812 16:58:02.478342 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.478346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.478360 (+ 14us) server_negotiation.cc:407] Connection header received
0812 16:58:02.478399 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.478403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.478458 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.478903 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.479495 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.480560 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.482828 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.483548 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.483687 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.488419 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.488433 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.488438 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.488476 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.492226 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.492905 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.492912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.492915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.493063 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.493565 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.493571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.493573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.493961 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.494938 (+ 977us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.495252 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.495588 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:02.503422 20927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: Bootstrap starting.
I20260812 16:58:02.507678 20927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:02.509143 20927 log.cc:824] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: Log is configured to *not* fsync() on all Append() calls
I20260812 16:58:02.509491 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.488786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:52692)
0812 16:58:02.489123 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.489149 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.489292 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.494005 (+ 4713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.494011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.494546 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.494752 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.494764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.497709 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.497716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.498557 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.498567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.499234 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.499239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.499476 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.500039 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.500055 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.502549 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.506449 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.506454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.506458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.506970 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.507352 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.507357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.507361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.507493 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.508235 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.508243 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.508416 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.508609 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.508995 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":5}
I20260812 16:58:02.510023 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.489429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.8.195.189:34925)
0812 16:58:02.489681 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.489691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.489731 (+ 40us) server_negotiation.cc:407] Connection header received
0812 16:58:02.489925 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.489935 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.490037 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.490168 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.491905 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.493529 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.495397 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.496184 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.496376 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.502260 (+ 5884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.502273 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.502276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.502315 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.505996 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.507005 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.507012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.507015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.507219 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.507765 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.507770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.507774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.508216 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.508515 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.509435 (+ 920us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.509699 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 16:58:02.510761 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.488798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.8.195.188:37643)
0812 16:58:02.493698 (+ 4900us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.493703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.493724 (+ 21us) server_negotiation.cc:407] Connection header received
0812 16:58:02.493768 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.493772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.493838 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.493943 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.494946 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.497582 (+ 2636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.498693 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.499098 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.499224 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.502793 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.502811 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.502822 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.502851 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.506322 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.507109 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.507113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.507115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.507229 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.507596 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.507604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.507607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.508045 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.508187 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.508725 (+ 538us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.509439 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":230,"server-negotiator.queue_time_us":4749,"thread_start_us":4531,"threads_started":1}
I20260812 16:58:02.514859 20927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: No bootstrap required, opened a new log
I20260812 16:58:02.515710 21251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: Bootstrap starting.
I20260812 16:58:02.515977 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:58:02.518860 21251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:02.519948 21251 log.cc:824] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: Log is configured to *not* fsync() on all Append() calls
I20260812 16:58:02.519992 20872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bb5ee8e9f2274a55bfda61fdf121800b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45656
I20260812 16:58:02.523430 21251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: No bootstrap required, opened a new log
I20260812 16:58:02.525476 20927 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.525866 20927 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5ee8e9f2274a55bfda61fdf121800b, State: Initialized, Role: FOLLOWER
I20260812 16:58:02.526436 20927 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.527527 21270 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.528000 21270 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.528791 20927 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:02.528947 20872 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:58:02.529220 20872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:02.532972 20872 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.534559 21251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.535027 21251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235d3aa566054f6bb878d246fb02827e, State: Initialized, Role: FOLLOWER
I20260812 16:58:02.535642 20872 leader_election.cc:288] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [CANDIDATE]: Term 1 election: Requested vote from peers 86db490bc8ef4eb1ae7369d798c5fe99 (127.8.195.189:34925), 235d3aa566054f6bb878d246fb02827e (127.8.195.188:37643)
I20260812 16:58:02.536355 21251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.541031 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.542452 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.544770 21251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:02.544731 21210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "235d3aa566054f6bb878d246fb02827e"
I20260812 16:58:02.545328 21210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:02.550427 21210 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb5ee8e9f2274a55bfda61fdf121800b in term 1.
I20260812 16:58:02.551493 20810 leader_election.cc:302] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235d3aa566054f6bb878d246fb02827e, bb5ee8e9f2274a55bfda61fdf121800b; no voters:
I20260812 16:58:02.553084 21270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 1 FOLLOWER]: Leader election won for term 1
W20260812 16:58:02.556718 21297 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:58:02.556984 21297 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:58:02.557624 21270 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [term 1 LEADER]: Becoming Leader. State: Replica: bb5ee8e9f2274a55bfda61fdf121800b, State: Running, Role: LEADER
I20260812 16:58:02.558244 21270 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
W20260812 16:58:02.565686 20820 tablet.cc:2369] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:58:02.572340 21313 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5ee8e9f2274a55bfda61fdf121800b. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.572850 21313 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:02.600032 21332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:58:02.576884 20769 master_runner.cc:186] Error getting master registration for 127.8.195.189:34925: OK, instance_id { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" instance_seqno: 1786553881875754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:58:02.606014 20769 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:58:02.608265 21101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } attempt: 5
I20260812 16:58:02.609726 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.567492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.8.195.190:36833)
0812 16:58:02.567693 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.567701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.575720 (+ 8019us) server_negotiation.cc:407] Connection header received
0812 16:58:02.577481 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.577489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.577603 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.577688 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.581135 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.584537 (+ 3402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.587325 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.588431 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.588555 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.597568 (+ 9013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.597576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.597581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.597632 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.602751 (+ 5119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.605676 (+ 2925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.605683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.605686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.605865 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.606454 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.606464 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.606471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.607108 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.607517 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.608443 (+ 926us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.608788 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:58:02.610121 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.561756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.8.195.189:34925)
0812 16:58:02.562005 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.562012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.574680 (+ 12668us) server_negotiation.cc:407] Connection header received
0812 16:58:02.575860 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.575869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.575969 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.576057 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.581299 (+ 5242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.582515 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.588851 (+ 6336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.589578 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.589696 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.599945 (+ 10249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.599952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.599957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.600011 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.603701 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.606446 (+ 2745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.606452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.606455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.606632 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.607150 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.607156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.607159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.607561 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.607826 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.609501 (+ 1675us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.609753 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 16:58:02.616367 21332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 16:58:02 dist-test-slave-z8x9 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:02.631958 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.608835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:37643 (local address 127.0.0.1:52714)
0812 16:58:02.609258 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:02.609288 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:02.609391 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:02.610687 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:02.610693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:02.611242 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:02.611489 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.611502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.612987 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.612994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.614431 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:02.614449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.615836 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.615842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:02.616076 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.617033 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:02.617056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:02.620052 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:02.628386 (+ 8334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.628391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.628395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.628943 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.629491 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:02.629499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:02.629506 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:02.629725 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:02.630559 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:02.630568 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:02.630795 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 16:58:02.630965 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 16:58:02.631367 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260812 16:58:02.633143 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.609606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.8.195.188:37643)
0812 16:58:02.609798 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.609804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.609821 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:58:02.609878 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.609884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.609961 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.610056 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.611656 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.612789 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.614605 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.615459 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.615591 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.620274 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.620281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.620287 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.620330 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.628221 (+ 7891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.629116 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.629121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.629125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.629316 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.629901 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.629909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.629915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.630388 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.632236 (+ 1848us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.632400 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.632710 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":70}
I20260812 16:58:02.636444 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.582803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.8.195.188:37643)
0812 16:58:02.583042 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.583048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.608880 (+ 25832us) server_negotiation.cc:407] Connection header received
0812 16:58:02.611295 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.611300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.611376 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.611452 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.614116 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.615274 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.617270 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.617927 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.618021 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.621810 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:02.621821 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:02.621833 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:02.621884 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:02.626491 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.634063 (+ 7572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.634068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.634074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.634224 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:02.634706 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:02.634714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:02.634720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:02.635084 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:02.635315 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.635887 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.636113 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":78}
I20260812 16:58:02.638342 21210 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [term 1 FOLLOWER]: Refusing update from remote peer bb5ee8e9f2274a55bfda61fdf121800b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:58:02.639425 21313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [LEADER]: Connected to new peer: Peer: permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:58:02.641620 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: Bootstrap starting.
I20260812 16:58:02.645927 21101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:02.647564 21101 log.cc:824] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: Log is configured to *not* fsync() on all Append() calls
I20260812 16:58:02.651726 21101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: No bootstrap required, opened a new log
I20260812 16:58:02.657850 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5ee8e9f2274a55bfda61fdf121800b. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.658183 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.665879 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.666173 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.673137 21345 mvcc.cc:205] Tried to move back new op lower bound from 7317724703245520896 to 7317724702965604352. Current Snapshot: MvccSnapshot[applied={T|T < 7317724703245520896}]
I20260812 16:58:02.676335 21313 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.676646 21313 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:02.681027 21332 catalog_manager.cc:1347] Generated new cluster ID: 63ae862d17164bbc941335223a8be933
I20260812 16:58:02.681164 21332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:58:02.683816 21354 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:58:02.680147 21101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.684544 21101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86db490bc8ef4eb1ae7369d798c5fe99, State: Initialized, Role: FOLLOWER
I20260812 16:58:02.685139 21101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } }
I20260812 16:58:02.701952 21357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.702610 21332 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:58:02.704644 21332 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:58:02.704905 21101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:02.705327 21037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:02.709277 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "86db490bc8ef4eb1ae7369d798c5fe99"
I20260812 16:58:02.711341 21357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.711776 21037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [term 1 FOLLOWER]: Refusing update from remote peer bb5ee8e9f2274a55bfda61fdf121800b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:58:02.715535 21038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb5ee8e9f2274a55bfda61fdf121800b in term 1.
I20260812 16:58:02.716923 21313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [LEADER]: Connected to new peer: Peer: permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:58:02.718811 21332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b: Generated new TSK 0
I20260812 16:58:02.720161 21332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 16:58:02.691149 21116 master_runner.cc:186] Error getting master registration for 127.8.195.189:34925: OK, instance_id { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" instance_seqno: 1786553881875754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:58:02.731017 21116 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:58:02.778971 21313 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.779407 21313 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5ee8e9f2274a55bfda61fdf121800b [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:02.783738 21357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5ee8e9f2274a55bfda61fdf121800b. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.784102 21357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.784559 21372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb5ee8e9f2274a55bfda61fdf121800b" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 36833 } } peers { permanent_uuid: "86db490bc8ef4eb1ae7369d798c5fe99" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 34925 } } peers { permanent_uuid: "235d3aa566054f6bb878d246fb02827e" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 37643 } } }
I20260812 16:58:02.785050 21372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:02.800412 21402 catalog_manager.cc:1259] Loaded cluster ID: 63ae862d17164bbc941335223a8be933
I20260812 16:58:02.800587 21402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: loading cluster ID for follower catalog manager: success
I20260812 16:58:02.804546 21402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: acquiring CA information for follower catalog manager: success
I20260812 16:58:02.811173 21402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86db490bc8ef4eb1ae7369d798c5fe99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 16:58:02.827773 8974 mini_kdc.cc:348] Time spent kdestroy: real 0.100s user 0.001s sys 0.035s
W20260812 16:58:02.841940 21262 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.833214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.8.195.190:36833)
0812 16:58:02.833648 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.833654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.839662 (+ 6008us) server_negotiation.cc:407] Connection header received
0812 16:58:02.840180 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.840188 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.840288 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.840370 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.841673 (+ 1303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45680: BlockingRecv error: recv got EOF from 127.0.0.1:45680 (error 108)
Metrics: {"server-negotiator.queue_time_us":286}
W20260812 16:58:02.844383 21111 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.832982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34622 (local address 127.8.195.189:34925)
0812 16:58:02.833469 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.833476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.842621 (+ 9145us) server_negotiation.cc:407] Connection header received
0812 16:58:02.842893 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.842901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.842983 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.843063 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.844144 (+ 1081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34622: BlockingRecv error: recv got EOF from 127.0.0.1:34622 (error 108)
Metrics: {"server-negotiator.queue_time_us":312}
W20260812 16:58:02.846626 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.844561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.8.195.188:37643)
0812 16:58:02.844750 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.844756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.845300 (+ 544us) server_negotiation.cc:407] Connection header received
0812 16:58:02.845422 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.845428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.845520 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.845627 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.846372 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52728: BlockingRecv error: recv got EOF from 127.0.0.1:52728 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:58:02.859619 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.850897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34636 (local address 127.8.195.189:34925)
0812 16:58:02.851243 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.851250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.852335 (+ 1085us) server_negotiation.cc:407] Connection header received
0812 16:58:02.852532 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.852538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.852755 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.852933 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:02.853502 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.854673 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.856679 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.857426 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.857549 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.858166 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:02.858904 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:02.859063 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.859165 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.859262 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
W20260812 16:58:02.875334 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.870686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.8.195.188:37643)
0812 16:58:02.870895 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.870901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.873745 (+ 2844us) server_negotiation.cc:407] Connection header received
0812 16:58:02.873967 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.873973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.874103 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.874225 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.875084 (+ 859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52732: BlockingRecv error: recv got EOF from 127.0.0.1:52732 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 16:58:02.893096 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.879896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34646 (local address 127.8.195.189:34925)
0812 16:58:02.880082 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.880089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.885957 (+ 5868us) server_negotiation.cc:407] Connection header received
0812 16:58:02.886398 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.886406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.886496 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.886567 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:02.887230 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.888419 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.890759 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.891488 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.891585 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.892222 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:02.892402 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:02.892470 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.892702 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.892788 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 16:58:02.923480 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.918809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.8.195.188:37643)
0812 16:58:02.919015 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.919022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.921839 (+ 2817us) server_negotiation.cc:407] Connection header received
0812 16:58:02.922051 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.922057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.922159 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.922275 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.923242 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52742: BlockingRecv error: recv got EOF from 127.0.0.1:52742 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:58:02.940673 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.930103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.8.195.189:34925)
0812 16:58:02.930346 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.930352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.933809 (+ 3457us) server_negotiation.cc:407] Connection header received
0812 16:58:02.934088 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.934094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.934195 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.934315 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:02.934825 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.936132 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.938234 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.939023 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.939228 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.939807 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:02.939954 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:02.940047 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.940231 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.940318 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260812 16:58:02.951349 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.948071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.8.195.188:37643)
0812 16:58:02.948237 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.948244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.949731 (+ 1487us) server_negotiation.cc:407] Connection header received
0812 16:58:02.950014 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.950020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.950118 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.950237 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.951104 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52744: BlockingRecv error: recv got EOF from 127.0.0.1:52744 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:58:02.966042 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.958738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.8.195.189:34925)
0812 16:58:02.958968 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.958976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.959552 (+ 576us) server_negotiation.cc:407] Connection header received
0812 16:58:02.959740 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.959746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.959846 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.959949 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:02.960432 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.961641 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.963753 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.964493 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.964620 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.965244 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:02.965388 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:02.965481 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.965661 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.965757 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260812 16:58:02.975077 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:02.972703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.8.195.188:37643)
0812 16:58:02.972976 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.972982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.973566 (+ 584us) server_negotiation.cc:407] Connection header received
0812 16:58:02.973770 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.973778 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.973877 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.973984 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:02.974822 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52756: BlockingRecv error: recv got EOF from 127.0.0.1:52756 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
I20260812 16:58:02.993672 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:02.986563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.8.195.189:34925)
0812 16:58:02.986745 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:02.986752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:02.987308 (+ 556us) server_negotiation.cc:407] Connection header received
0812 16:58:02.987502 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:02.987511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:02.987605 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:02.987721 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:02.988187 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.989403 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.991413 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:02.992125 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:02.992286 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:02.992744 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:02.992925 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:02.993021 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:02.993218 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 16:58:02.993313 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 16:58:03.010190 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.007501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.8.195.188:37643)
0812 16:58:03.007745 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.007752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.008356 (+ 604us) server_negotiation.cc:407] Connection header received
0812 16:58:03.008580 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.008586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.008684 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.008797 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.009908 (+ 1111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52758: BlockingRecv error: recv got EOF from 127.0.0.1:52758 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 16:58:03.039047 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:03.031181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.8.195.189:34925)
0812 16:58:03.031405 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.031412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.032053 (+ 641us) server_negotiation.cc:407] Connection header received
0812 16:58:03.032290 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.032296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.032391 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.032496 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:03.033058 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.034411 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.036553 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.037300 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.037433 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:03.038069 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:03.038264 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:03.038357 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:03.038576 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 16:58:03.038680 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 16:58:03.051476 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.047101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.8.195.188:37643)
0812 16:58:03.047304 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.047311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.049748 (+ 2437us) server_negotiation.cc:407] Connection header received
0812 16:58:03.049972 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.049977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.050072 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.050263 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.051213 (+ 950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52772: BlockingRecv error: recv got EOF from 127.0.0.1:52772 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:58:03.099120 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:03.091284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.8.195.189:34925)
0812 16:58:03.091530 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.091539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.092116 (+ 577us) server_negotiation.cc:407] Connection header received
0812 16:58:03.092383 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.092388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.092511 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.092619 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:03.093233 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.094397 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.096541 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.097356 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.097493 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:03.098219 (+ 726us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:03.098373 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:03.098478 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:03.098670 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 16:58:03.098763 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 16:58:03.110692 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.107859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.8.195.188:37643)
0812 16:58:03.108087 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.108095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.109028 (+ 933us) server_negotiation.cc:407] Connection header received
0812 16:58:03.109266 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.109272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.109372 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.109485 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.110422 (+ 937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52780: BlockingRecv error: recv got EOF from 127.0.0.1:52780 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 16:58:03.185484 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:03.178953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.8.195.189:34925)
0812 16:58:03.179227 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.179237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.179616 (+ 379us) server_negotiation.cc:407] Connection header received
0812 16:58:03.179835 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.179839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.179910 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.179995 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:03.180539 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.181430 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.183491 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.183989 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.184106 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:03.184650 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:03.184760 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:03.184844 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:03.185038 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 16:58:03.185106 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 16:58:03.193820 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.191579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.8.195.188:37643)
0812 16:58:03.191826 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.191834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.192218 (+ 384us) server_negotiation.cc:407] Connection header received
0812 16:58:03.192423 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.192427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.192514 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.192593 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.193597 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52782: BlockingRecv error: recv got EOF from 127.0.0.1:52782 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 16:58:03.331141 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:03.325930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.8.195.189:34925)
0812 16:58:03.326190 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.326198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.326517 (+ 319us) server_negotiation.cc:407] Connection header received
0812 16:58:03.326757 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.326761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.326843 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.326946 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:03.327374 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.328086 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.329424 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.329867 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.330004 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:03.330415 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:03.330529 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:03.330627 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:03.330786 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 16:58:03.330867 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 16:58:03.347465 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.338771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.8.195.188:37643)
0812 16:58:03.339054 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.339063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.345813 (+ 6750us) server_negotiation.cc:407] Connection header received
0812 16:58:03.346047 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.346052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.346131 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.346234 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.347208 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52790: BlockingRecv error: recv got EOF from 127.0.0.1:52790 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 16:58:03.613619 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:03.608507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.8.195.189:34925)
0812 16:58:03.608745 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.608751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.608947 (+ 196us) server_negotiation.cc:407] Connection header received
0812 16:58:03.609203 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.609207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.609293 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.609401 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:03.609849 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.610574 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.611912 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:03.612314 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:03.612451 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:03.612863 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:03.612968 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:03.613072 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:03.613268 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 16:58:03.613350 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 16:58:03.621629 21266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:03.619709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.8.195.188:37643)
0812 16:58:03.619944 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:03.619950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:03.620152 (+ 202us) server_negotiation.cc:407] Connection header received
0812 16:58:03.620394 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:03.620399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:03.620471 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:03.620581 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:03.621407 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52802: BlockingRecv error: recv got EOF from 127.0.0.1:52802 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 16:58:03.688325 21354 catalog_manager.cc:1259] Loaded cluster ID: 63ae862d17164bbc941335223a8be933
I20260812 16:58:03.688452 21354 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: loading cluster ID for follower catalog manager: success
I20260812 16:58:03.691089 21354 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: acquiring CA information for follower catalog manager: success
I20260812 16:58:03.692973 21354 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 235d3aa566054f6bb878d246fb02827e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:58:04.143325 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:04.138105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.8.195.189:34925)
0812 16:58:04.138708 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:04.138713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:04.138733 (+ 20us) server_negotiation.cc:407] Connection header received
0812 16:58:04.138896 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:04.138900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:04.138969 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:04.139056 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:04.139518 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.140287 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.141634 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.142029 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.142164 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:04.142565 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:04.142665 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:04.142762 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:04.142923 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 16:58:04.143004 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":354,"threads_started":1}
I20260812 16:58:04.156425 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:04.149996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.8.195.188:37643)
0812 16:58:04.151184 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:04.151189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:04.151204 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:58:04.151266 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:04.151270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:04.151441 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:04.151567 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:04.151964 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.152657 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.154050 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.154485 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.154612 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:04.155218 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:04.155776 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:04.155891 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:04.156021 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 16:58:04.156088 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":941,"threads_started":1}
I20260812 16:58:04.171818 21638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:04.163000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.8.195.190:36833)
0812 16:58:04.163817 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:04.163824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:04.164479 (+ 655us) server_negotiation.cc:407] Connection header received
0812 16:58:04.164637 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:04.164645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:04.164919 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:04.165141 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:04.166525 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.167463 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.169702 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.170106 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.170254 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:04.170736 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:04.171116 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:04.171222 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:04.171395 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 16:58:04.171484 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":511,"threads_started":1}
I20260812 16:58:04.173102 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:04.163130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.8.195.189:34925)
0812 16:58:04.163360 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:04.163369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:04.166491 (+ 3122us) server_negotiation.cc:407] Connection header received
0812 16:58:04.166770 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:04.166776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:04.166860 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:04.166956 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:04.168096 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.168809 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.171097 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.171717 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.171857 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:04.172338 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:04.172468 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:04.172552 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:04.172734 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 16:58:04.172817 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 16:58:04.173511 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:04.164242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.8.195.188:37643)
0812 16:58:04.164466 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:04.164474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:04.166839 (+ 2365us) server_negotiation.cc:407] Connection header received
0812 16:58:04.167027 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:04.167033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:04.167109 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:04.167205 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:04.167920 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.169003 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.171141 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:04.171842 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:04.171965 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:04.172630 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:04.172741 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:04.172843 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:04.173190 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 16:58:04.173271 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:58:04.181782 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20769
I20260812 16:58:04.183045 20916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:04.295961 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20769
I20260812 16:58:04.332144 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 20942
I20260812 16:58:04.333396 21088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:04.443449 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 20942
I20260812 16:58:04.471863 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21116
I20260812 16:58:04.473062 21244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:04.581853 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21116
2026-08-12T16:58:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5493 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 16:58:04 dist-test-slave-z8x9 krb5kdc[21655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 16:58:04 dist-test-slave-z8x9 krb5kdc[21655](info): set up 2 sockets
Aug 12 16:58:04 dist-test-slave-z8x9 krb5kdc[21655](info): commencing operation
krb5kdc: starting...
W20260812 16:58:06.039752 8974 mini_kdc.cc:121] Time spent starting KDC: real 1.409s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/test-user.keytab.
Aug 12 16:58:06 dist-test-slave-z8x9 krb5kdc[21655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T16:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T16:58:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.195.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
Entry for principal kudu/127.8.195.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab.
I20260812 16:58:06.407197 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42063
--webserver_interface=127.8.195.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 16:58:06.769986 21671 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:06.770359 21671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:06.770504 21671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:06.770632 21671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:06.777915 21671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:06.778007 21671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:06.778112 21671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:06.778188 21671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:06.778278 21671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:06.788071 21671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.190.keytab
--principal=kudu/127.8.195.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.190:42063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.195.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:06.789474 21671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:06.790877 21671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:06.802735 21677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:06.802843 21687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:06.802798 21682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:06.803623 21671 server_base.cc:1034] running on GCE node
I20260812 16:58:06.804515 21671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:06.806005 21671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:06.807229 21671 hybrid_clock.cc:648] HybridClock initialized: now 1786553886807168 us; error 30 us; skew 500 ppm
I20260812 16:58:06.807582 21671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:06 dist-test-slave-z8x9 krb5kdc[21655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:06.810925 21671 init.cc:377] Logged in from keytab as kudu/127.8.195.190@KRBTEST.COM (short username kudu)
I20260812 16:58:06.812450 21671 webserver.cc:460] Webserver started at http://127.8.195.190:37497/ using document root <none> and password file <none>
I20260812 16:58:06.813143 21671 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:06.813345 21671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:06.813678 21671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:06.816593 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data/instance:
uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca"
format_stamp: "Formatted at 2026-08-12 16:58:06 on dist-test-slave-z8x9"
server_key: "bec213bd8784dc40e4503562b7923bac"
server_key_iv: "2e8ff63328e9a42aa69681d8c2f20483"
server_key_version: "encryptionkey@0"
I20260812 16:58:06.817480 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance:
uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca"
format_stamp: "Formatted at 2026-08-12 16:58:06 on dist-test-slave-z8x9"
server_key: "bec213bd8784dc40e4503562b7923bac"
server_key_iv: "2e8ff63328e9a42aa69681d8c2f20483"
server_key_version: "encryptionkey@0"
I20260812 16:58:06.822696 21671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:58:06.826139 21705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:06.827431 21671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 16:58:06.827621 21671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal
uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca"
format_stamp: "Formatted at 2026-08-12 16:58:06 on dist-test-slave-z8x9"
server_key: "bec213bd8784dc40e4503562b7923bac"
server_key_iv: "2e8ff63328e9a42aa69681d8c2f20483"
server_key_version: "encryptionkey@0"
I20260812 16:58:06.827874 21671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:06.852366 21671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:06.856590 21671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:06.856884 21671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:06.875926 21671 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.190:42063
I20260812 16:58:06.875962 21821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.190:42063 every 8 connection(s)
I20260812 16:58:06.877308 21671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/data/info.pb
I20260812 16:58:06.880965 21822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:06.883289 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21671
I20260812 16:58:06.883486 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-0/wal/instance
I20260812 16:58:06.884049 8974 external_mini_cluster.cc:1428] Setting key 94e83997adaef66ace7a1f489db81186
I20260812 16:58:06.890621 21822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:06 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:06.917290 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.894631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.8.195.190:42063)
0812 16:58:06.896539 (+ 1908us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:06.896545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:06.896586 (+ 41us) server_negotiation.cc:407] Connection header received
0812 16:58:06.898380 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:06.898391 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:06.898772 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:06.899017 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:06.900208 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:06.901851 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:06.904157 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:06.904726 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:06.904885 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:06.909502 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:06.909511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:06.909524 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:06.909559 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:06.912632 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:06.913600 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:06.913606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:06.913608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:06.913804 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:06.914292 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:06.914296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:06.914298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:06.914820 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:06.914993 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:06.915779 (+ 786us) server_negotiation.cc:299] Negotiation successful
0812 16:58:06.916079 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1706,"spinlock_wait_cycles":15232,"thread_start_us":1455,"threads_started":1}
I20260812 16:58:06.917339 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.894631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42063 (local address 127.0.0.1:44286)
0812 16:58:06.895582 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:06.895635 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:06.897974 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:06.899037 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:06.899042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:06.899624 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:06.899975 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:06.899992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:06.902019 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:06.902027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:06.903983 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:06.904002 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:06.904980 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:06.904990 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:06.905253 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:06.906018 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:06.906052 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:06.909259 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:06.912793 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:06.912809 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:06.912825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:06.913475 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:06.913932 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:06.913939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:06.913945 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:06.914174 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:06.914973 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:06.914984 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:06.915278 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 16:58:06.915638 (+ 360us) client_negotiation.cc:240] Negotiation successful
0812 16:58:06.916253 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":51,"thread_start_us":391,"threads_started":1}
I20260812 16:58:06.920796 21822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:06.922024 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.921613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44154)
0812 16:58:06.921787 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:06.921913 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 16:58:06.922305 21706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.195.189@KRBTEST.COM; defaulting to no policy
I20260812 16:58:06.924068 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.923659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44158)
0812 16:58:06.923833 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:06.923916 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:58:06.924314 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:39371: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
Entry for principal kudu/127.8.195.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab.
I20260812 16:58:06.966789 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:39371
--webserver_interface=127.8.195.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:58:06.973762 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } attempt: 1
I20260812 16:58:06.975126 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.974572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44160)
0812 16:58:06.974814 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:06.974976 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 16:58:06.976244 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:06.975854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44174)
0812 16:58:06.976014 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:06.976097 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 16:58:06.976470 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:39371: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
I20260812 16:58:07.044755 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } attempt: 2
I20260812 16:58:07.046167 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.045577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44178)
0812 16:58:07.045850 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.045974 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 16:58:07.047361 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.046828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44186)
0812 16:58:07.047016 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.047138 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 16:58:07.047598 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:39371: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
I20260812 16:58:07.138907 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } attempt: 3
I20260812 16:58:07.140340 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.139709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44190)
0812 16:58:07.139987 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.140131 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 16:58:07.141705 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.141147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44200)
0812 16:58:07.141381 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.141492 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260812 16:58:07.141939 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:39371: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
I20260812 16:58:07.285280 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } attempt: 4
I20260812 16:58:07.286764 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.286034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44208)
0812 16:58:07.286426 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.286559 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260812 16:58:07.287947 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.287520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44210)
0812 16:58:07.287680 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.287786 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 16:58:07.288172 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.189:39371: Network error: Client connection negotiation failed: client connection to 127.8.195.189:39371: connect: Connection refused (error 111)
W20260812 16:58:07.315665 21834 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:07.316002 21834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:07.316077 21834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:07.316162 21834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:07.324299 21834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:07.324400 21834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:07.324457 21834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:07.324501 21834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:07.324565 21834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:07.334954 21834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.189.keytab
--principal=kudu/127.8.195.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.189:39371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.195.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:07.336232 21834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:07.337630 21834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:07.349695 21844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:07.349745 21849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:07.349647 21845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:07.350687 21834 server_base.cc:1034] running on GCE node
I20260812 16:58:07.351576 21834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:07.353005 21834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:07.354233 21834 hybrid_clock.cc:648] HybridClock initialized: now 1786553887354178 us; error 72 us; skew 500 ppm
I20260812 16:58:07.354588 21834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:07.358050 21834 init.cc:377] Logged in from keytab as kudu/127.8.195.189@KRBTEST.COM (short username kudu)
I20260812 16:58:07.359661 21834 webserver.cc:460] Webserver started at http://127.8.195.189:35337/ using document root <none> and password file <none>
I20260812 16:58:07.360388 21834 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:07.360512 21834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:07.360855 21834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:07.363996 21834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data/instance:
uuid: "cdcb508ad10149aa849ea0ca51fdf751"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "399732848eb314176aedffbf5b08443b"
server_key_iv: "d8b909578c93fb1a59f288e7b52117a5"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.364795 21834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal/instance:
uuid: "cdcb508ad10149aa849ea0ca51fdf751"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "399732848eb314176aedffbf5b08443b"
server_key_iv: "d8b909578c93fb1a59f288e7b52117a5"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.370204 21834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 16:58:07.374284 21865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:07.375711 21834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260812 16:58:07.375943 21834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal
uuid: "cdcb508ad10149aa849ea0ca51fdf751"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "399732848eb314176aedffbf5b08443b"
server_key_iv: "d8b909578c93fb1a59f288e7b52117a5"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.376214 21834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:07.390337 21834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:07.394939 21834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:07.395256 21834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:07.415027 21834 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.189:39371
I20260812 16:58:07.415035 21958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.189:39371 every 8 connection(s)
I20260812 16:58:07.416443 21834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/data/info.pb
I20260812 16:58:07.420280 21959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:07.422888 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21834
I20260812 16:58:07.423076 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-1/wal/instance
I20260812 16:58:07.423619 8974 external_mini_cluster.cc:1428] Setting key 13bd18aea4993e3d40c7d59571226e11
I20260812 16:58:07.429474 21959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:07.461583 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.433016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.8.195.190:42063)
0812 16:58:07.441705 (+ 8689us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:07.441712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:07.442383 (+ 671us) server_negotiation.cc:407] Connection header received
0812 16:58:07.444720 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:07.444729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:07.444830 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:07.444964 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:07.446605 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.447773 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.449365 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.450135 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.450351 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.454009 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:07.454018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:07.454026 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:07.454086 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:07.458042 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.458926 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.458935 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.458942 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.459107 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.459527 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.459535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.459541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.459959 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:07.460143 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:07.460846 (+ 703us) server_negotiation.cc:299] Negotiation successful
0812 16:58:07.461097 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8495,"thread_start_us":8334,"threads_started":1}
I20260812 16:58:07.462275 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.433486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42063 (local address 127.0.0.1:44294)
0812 16:58:07.442145 (+ 8659us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.442205 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:07.444417 (+ 2212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:07.445219 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:07.445240 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:07.445808 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:07.446371 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.446387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.447937 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.447941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.449141 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.449154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.450428 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.450433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.450635 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.451469 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:07.451497 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:07.453746 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:07.458200 (+ 4454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.458206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.458219 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.458764 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.459224 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.459228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.459230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.459399 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.460097 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:07.460102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:07.460426 (+ 324us) client_negotiation.cc:769] Sending connection context
0812 16:58:07.460672 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 16:58:07.461067 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8318,"thread_start_us":8094,"threads_started":1}
WARNING: no policy specified for kudu/127.8.195.188@KRBTEST.COM; defaulting to no policy
I20260812 16:58:07.464067 21959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.195.188@KRBTEST.COM" created.
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:07.481120 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.464807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44224)
0812 16:58:07.465008 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.465026 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:07.465115 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:07.466385 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:07.466391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:07.466905 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:07.467120 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.467135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.468533 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.468541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.470018 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.470046 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.470859 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.470872 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.471103 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.471757 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:07.471786 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:07.474290 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:07.478641 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.478646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.478648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.479020 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.479442 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.479446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.479448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.479568 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.480292 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:07.480296 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:07.480413 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 16:58:07.480522 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 16:58:07.480766 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 16:58:07.481670 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.465124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.8.195.189:39371)
0812 16:58:07.465882 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:07.465892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:07.465928 (+ 36us) server_negotiation.cc:407] Connection header received
0812 16:58:07.465989 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:07.466000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:07.466222 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:07.466344 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:07.467327 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.468385 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.470179 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.470708 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.470806 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.474520 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:07.474537 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:07.474556 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:07.474598 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:07.478506 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.479149 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.479160 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.479166 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.479322 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.479649 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.479656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.479662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.480133 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:07.480354 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:07.480742 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 16:58:07.481025 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":412,"threads_started":1}
I20260812 16:58:07.483886 21959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:07.485255 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.484660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57014)
0812 16:58:07.484857 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.485005 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 16:58:07.485544 21868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.487156 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.486733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57018)
0812 16:58:07.486906 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.487013 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 16:58:07.487404 21959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
Entry for principal kudu/127.8.195.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab.
I20260812 16:58:07.510154 8974 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
/tmp/dist-test-taskKFRmA3/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:38231
--webserver_interface=127.8.195.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.195.148:40405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 16:58:07.536772 21959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 1
I20260812 16:58:07.538235 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.537620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57020)
0812 16:58:07.537865 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.538017 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 16:58:07.539480 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.538972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57034)
0812 16:58:07.539165 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.539290 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 16:58:07.539713 21959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.587431 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } attempt: 5
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:07.602936 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.588355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.8.195.189:39371)
0812 16:58:07.588556 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:07.588564 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:07.588591 (+ 27us) server_negotiation.cc:407] Connection header received
0812 16:58:07.588798 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:07.588802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:07.588883 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:07.588966 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:07.590081 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.590786 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.592562 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.593024 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.593121 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.597271 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:07.597279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:07.597283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:07.597317 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:07.600129 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.600974 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.600978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.600980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.601104 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:07.601572 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:07.601575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:07.601577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:07.601887 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:07.601999 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:07.602507 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 16:58:07.602681 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 16:58:07.603180 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.588155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44232)
0812 16:58:07.588475 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.588502 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:07.588644 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:07.589043 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:07.589048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:07.589664 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:07.589910 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.589924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.590936 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.590942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.592380 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.592402 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.593221 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.593229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:07.593473 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:07.594180 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:07.594203 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:07.596982 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:07.600283 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.600292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.600298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.600847 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.601233 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:07.601238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:07.601241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:07.601446 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:07.602014 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:07.602020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:07.602223 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 16:58:07.602396 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 16:58:07.602728 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 16:58:07.604348 21822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:07.605674 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.605274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57038)
0812 16:58:07.605452 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.605532 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 16:58:07.606606 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.606275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57042)
0812 16:58:07.606413 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.606506 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 16:58:07.606809 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.607946 21959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 2
I20260812 16:58:07.608858 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.608419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57044)
0812 16:58:07.608598 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.608692 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 16:58:07.609846 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.609540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57050)
0812 16:58:07.609668 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.609735 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 16:58:07.610124 21959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.658043 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 1
I20260812 16:58:07.659220 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.658735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57060)
0812 16:58:07.658953 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.659053 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 16:58:07.660423 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.660084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57066)
0812 16:58:07.660226 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.660317 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 16:58:07.660673 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.701370 21959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 3
I20260812 16:58:07.702690 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.702173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57080)
0812 16:58:07.702418 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.702522 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 16:58:07.703905 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.703463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57090)
0812 16:58:07.703706 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.703788 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260812 16:58:07.704176 21959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.728907 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 2
I20260812 16:58:07.730383 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.729782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57092)
0812 16:58:07.730049 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.730145 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 16:58:07.731498 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.731104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57102)
0812 16:58:07.731283 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.731356 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 16:58:07.731766 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.838039 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 3
I20260812 16:58:07.839296 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.838781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57110)
0812 16:58:07.838997 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.839097 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 16:58:07.840492 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.840087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57118)
0812 16:58:07.840306 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.840385 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260812 16:58:07.840806 21822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
I20260812 16:58:07.847436 21959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 4
I20260812 16:58:07.848639 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.848149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57126)
0812 16:58:07.848383 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.848481 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 16:58:07.849716 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.849380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57132)
0812 16:58:07.849530 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.849609 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 16:58:07.849918 21959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.195.188:38231: Network error: Client connection negotiation failed: client connection to 127.8.195.188:38231: connect: Connection refused (error 111)
W20260812 16:58:07.871861 21980 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 16:58:07.872273 21980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 16:58:07.872403 21980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 16:58:07.872545 21980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 16:58:07.880435 21980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 16:58:07.880556 21980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 16:58:07.880682 21980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 16:58:07.880771 21980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 16:58:07.880872 21980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 16:58:07.891089 21980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.195.148:40405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.195.190:42063,127.8.195.189:39371,127.8.195.188:38231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/krb5kdc/kudu_127.8.195.188.keytab
--principal=kudu/127.8.195.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.195.188:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.195.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 15:17:36 UTC on ccycloud.quasar-sxjchh.root.comops.site
build id 4582766
ASAN enabled
I20260812 16:58:07.892383 21980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 16:58:07.893757 21980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 16:58:07.905972 21992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:07.905905 21991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 16:58:07.906050 21997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 16:58:07.907019 21980 server_base.cc:1034] running on GCE node
I20260812 16:58:07.907836 21980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 16:58:07.909294 21980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 16:58:07.910588 21980 hybrid_clock.cc:648] HybridClock initialized: now 1786553887910532 us; error 48 us; skew 500 ppm
I20260812 16:58:07.910955 21980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 16:58:07.914703 21980 init.cc:377] Logged in from keytab as kudu/127.8.195.188@KRBTEST.COM (short username kudu)
I20260812 16:58:07.916472 21980 webserver.cc:460] Webserver started at http://127.8.195.188:38887/ using document root <none> and password file <none>
I20260812 16:58:07.917279 21980 fs_manager.cc:359] Metadata directory not provided
I20260812 16:58:07.917411 21980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 16:58:07.917770 21980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 16:58:07.920781 21980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data/instance:
uuid: "a0b9cdd800c546e9b4738d16cbdc3128"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "84ac1c0aad5d40fe1b8b9015463df405"
server_key_iv: "071e1f39016552a1f97bc81a5e120fd3"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.921638 21980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal/instance:
uuid: "a0b9cdd800c546e9b4738d16cbdc3128"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "84ac1c0aad5d40fe1b8b9015463df405"
server_key_iv: "071e1f39016552a1f97bc81a5e120fd3"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.927021 21980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 16:58:07.930552 22014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 16:58:07.931950 21980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 16:58:07.932134 21980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data,/tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal
uuid: "a0b9cdd800c546e9b4738d16cbdc3128"
format_stamp: "Formatted at 2026-08-12 16:58:07 on dist-test-slave-z8x9"
server_key: "84ac1c0aad5d40fe1b8b9015463df405"
server_key_iv: "071e1f39016552a1f97bc81a5e120fd3"
server_key_version: "encryptionkey@0"
I20260812 16:58:07.932401 21980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 16:58:07.955147 21980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 16:58:07.959692 21980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 16:58:07.959930 21980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 16:58:07.978871 21980 rpc_server.cc:307] RPC server started. Bound to: 127.8.195.188:38231
I20260812 16:58:07.978901 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.195.188:38231 every 8 connection(s)
I20260812 16:58:07.980347 21980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/data/info.pb
I20260812 16:58:07.984185 22119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 16:58:07.986138 8974 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu as pid 21980
I20260812 16:58:07.986359 8974 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKFRmA3/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786553491558846-8974-0/minicluster-data/master-2/wal/instance
I20260812 16:58:07.986821 8974 external_mini_cluster.cc:1428] Setting key ae86362087776ad431a1ba3f6c17de2f
I20260812 16:58:07.993753 22119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 16:58:07 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
I20260812 16:58:08.011912 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.988614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.8.195.188:38231)
0812 16:58:07.989483 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:07.989499 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:07.989551 (+ 52us) server_negotiation.cc:407] Connection header received
0812 16:58:07.991167 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:07.991197 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:07.991723 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:07.992122 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:07.993475 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.995467 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.997061 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:07.998023 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:07.998214 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.003184 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.003191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.003202 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.003234 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.007073 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.007809 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.007819 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.007829 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.008030 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.008634 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.008641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.008645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.009404 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.009732 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.010095 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.010550 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":324,"threads_started":1}
I20260812 16:58:08.013804 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.996729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44300 (local address 127.8.195.190:42063)
0812 16:58:07.997297 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:07.997304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:07.997867 (+ 563us) server_negotiation.cc:407] Connection header received
0812 16:58:07.998055 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:07.998063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:07.998166 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:07.998277 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:07.999687 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.000847 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.002645 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.003262 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.003360 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.007425 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.007431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.007435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.007468 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.010902 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.011623 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.011628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.011631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.011805 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.012262 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.012265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.012267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.012510 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.012681 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.013119 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.013362 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":238,"threads_started":1}
I20260812 16:58:08.013809 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:07.996732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.190:42063 (local address 127.0.0.1:44300)
0812 16:58:07.997705 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:07.997757 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:07.997923 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:07.998371 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:07.998380 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:07.999291 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:07.999526 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:07.999535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.001014 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.001018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.002484 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.002496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.003457 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.003463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.003668 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.004413 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:08.004436 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:08.007239 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:08.011030 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.011040 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.011045 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.011482 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.011952 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.011959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.011964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.012123 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.012644 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:08.012652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:08.012831 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 16:58:08.013007 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 16:58:08.013344 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":578,"threads_started":1}
I20260812 16:58:08.015272 22119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:08.018036 21822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 4
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.190@KRBTEST.COM
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.190@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:08.033411 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.018637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.8.195.188:38231)
0812 16:58:08.018783 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.018799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.018878 (+ 79us) server_negotiation.cc:407] Connection header received
0812 16:58:08.019064 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.019069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.019130 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.019224 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.019899 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.020889 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.022468 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.023061 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.023256 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.027214 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.027221 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.027226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.027267 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.030389 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.031259 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.031267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.031270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.031445 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.031900 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.031924 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.031927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.032227 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.032388 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.032803 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.033007 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 16:58:08.033468 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.018596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57148)
0812 16:58:08.018814 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:08.018831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:08.018929 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:08.019250 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:08.019255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:08.019624 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:08.019760 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.019768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.021022 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.021026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.022318 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.022334 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.023267 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.023273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.023471 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.024018 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:08.024035 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:08.026997 (+ 2962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:08.030518 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.030524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.030528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.031150 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.031559 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.031566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.031569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.031798 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.032357 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:08.032362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:08.032556 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 16:58:08.032699 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 16:58:08.032997 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 16:58:08.033947 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.016015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.8.195.189:39371)
0812 16:58:08.016182 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.016186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.016201 (+ 15us) server_negotiation.cc:407] Connection header received
0812 16:58:08.016340 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.016343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.016399 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.016476 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.017386 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.018415 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.020237 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.020888 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.021024 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.026203 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.026210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.026215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.026267 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.030325 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.031265 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.031273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.031280 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.031481 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.032018 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.032023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.032026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.032499 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.032630 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.033320 (+ 690us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.033539 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 16:58:08.034816 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.015917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.189:39371 (local address 127.0.0.1:44240)
0812 16:58:08.016110 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:08.016131 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:08.016225 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:08.016539 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:08.016545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:08.017013 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:08.017236 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.017252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.018606 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.018612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.020071 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.020091 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.022385 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.022393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.022637 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.023237 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:08.023273 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:08.025978 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:08.030501 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.030506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.030510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.031066 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.031670 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.031674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.031678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.031874 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.032735 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:08.032740 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:08.032924 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 16:58:08.033074 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 16:58:08.034448 (+ 1374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260812 16:58:08.035444 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.015011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.8.195.190:42063)
0812 16:58:08.015439 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.015447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.015469 (+ 22us) server_negotiation.cc:407] Connection header received
0812 16:58:08.015505 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.015508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.015569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.015670 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.017343 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.018462 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.021239 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.021679 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.021768 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.027599 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.027607 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.027612 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.027653 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.031536 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.032444 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.032451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.032456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.032604 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.033098 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.033104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.033107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.033517 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.033828 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.034955 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.035216 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260812 16:58:08.035815 22119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } has no permanent_uuid. Determining permanent_uuid...
I20260812 16:58:08.040226 21822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Bootstrap starting.
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.188@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:08.044591 21822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:08.045915 21822 log.cc:824] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Log is configured to *not* fsync() on all Append() calls
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.195.189@KRBTEST.COM
I20260812 16:58:08.048827 21822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: No bootstrap required, opened a new log
I20260812 16:58:08.052409 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.036658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57164)
0812 16:58:08.036836 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:08.036856 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:08.036963 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:08.037651 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:08.037655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:08.038165 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:08.038340 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.038351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.039759 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.039766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.040676 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.040693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.041629 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.041636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.041877 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.042493 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:08.042509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:08.045219 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:08.049036 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.049039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.049042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.049626 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.050350 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.050356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.050359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.050536 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.051268 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:08.051275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:08.051449 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 16:58:08.051641 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 16:58:08.051996 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 16:58:08.052639 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.036968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.8.195.188:38231)
0812 16:58:08.037293 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.037299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.037316 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:58:08.037358 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.037363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.037430 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.037520 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.038523 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.039625 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.040810 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.041456 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.041588 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.045421 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.045427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.045432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.045469 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.048891 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.049761 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.049767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.049770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.049931 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.050672 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.050678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.050682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.051117 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.051274 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.051797 (+ 523us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.052175 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":181}
I20260812 16:58:08.055085 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.037609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.8.195.189:39371)
0812 16:58:08.037785 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.037789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.037802 (+ 13us) server_negotiation.cc:407] Connection header received
0812 16:58:08.037841 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.037845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.037895 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.037999 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.039522 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.040576 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.042133 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.042840 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.042978 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.048289 (+ 5311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.048297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.048301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.048343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.051677 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.052573 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.052580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.052586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.052740 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.053281 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.053284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.053286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.053569 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.054603 (+ 1034us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.054695 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.054857 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 16:58:08.058588 22119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: Bootstrap starting.
I20260812 16:58:08.059733 8974 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 16:58:08.059674 21822 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.060070 21822 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca, State: Initialized, Role: FOLLOWER
I20260812 16:58:08.060618 21822 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.061707 22131 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.062250 22131 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.062592 21761 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44304
I20260812 16:58:08.062726 22119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:08.063050 21822 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:08.063090 21761 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 16:58:08.063321 21761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:08.064054 22119 log.cc:824] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: Log is configured to *not* fsync() on all Append() calls
I20260812 16:58:08.066495 21761 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.068043 21761 leader_election.cc:288] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [CANDIDATE]: Term 1 election: Requested vote from peers cdcb508ad10149aa849ea0ca51fdf751 (127.8.195.189:39371), a0b9cdd800c546e9b4738d16cbdc3128 (127.8.195.188:38231)
I20260812 16:58:08.086434 22119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: No bootstrap required, opened a new log
I20260812 16:58:08.105928 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.078574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.8.195.190:42063)
0812 16:58:08.079021 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.079029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.082871 (+ 3842us) server_negotiation.cc:407] Connection header received
0812 16:58:08.083912 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.083921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.084015 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.084095 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.086404 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.087635 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.089778 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.090442 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.090540 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.096444 (+ 5904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.096454 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.096461 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.096513 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.101128 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.102412 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.102421 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.102425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.102612 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.103102 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.103107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.103112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.103577 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.103902 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.105198 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.105503 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
W20260812 16:58:08.109862 21719 tablet.cc:2369] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:58:08.109985 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.078341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.8.195.189:39371)
0812 16:58:08.078837 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.078846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.082526 (+ 3680us) server_negotiation.cc:407] Connection header received
0812 16:58:08.083157 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.083166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.083274 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.083359 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.089051 (+ 5692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.090198 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.093024 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.093765 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.093886 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.098910 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.098918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.098924 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.098990 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.103580 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.104714 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.104720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.104755 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.105123 (+ 368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.106619 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.106624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.106627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.107561 (+ 934us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.108064 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.109339 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.109610 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318}
I20260812 16:58:08.112569 22119 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.113392 22119 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b9cdd800c546e9b4738d16cbdc3128, State: Initialized, Role: FOLLOWER
I20260812 16:58:08.114130 22119 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
W20260812 16:58:08.127820 22169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:58:08.127970 22169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:58:08.131111 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.080491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.8.195.188:38231)
0812 16:58:08.080720 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.080729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.093965 (+ 13236us) server_negotiation.cc:407] Connection header received
0812 16:58:08.094638 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.094648 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.095230 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.095335 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.113329 (+ 17994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.114470 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.116539 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.117229 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.117385 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.121952 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.121963 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.121970 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.122026 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.126234 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.127204 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.127214 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.127218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.127562 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.128178 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.128183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.128186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.128984 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.129547 (+ 563us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.129835 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.130382 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 16:58:08.121377 21671 master_runner.cc:186] Error getting master registration for 127.8.195.189:39371: OK, instance_id { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" instance_seqno: 1786553887409688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:58:08.134140 21671 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:58:08.137977 22170 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.138419 22170 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.140260 22119 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:08.140306 22071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a0b9cdd800c546e9b4738d16cbdc3128"
I20260812 16:58:08.140815 22071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:08.145792 22071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca in term 1.
I20260812 16:58:08.146570 21709 leader_election.cc:302] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0b9cdd800c546e9b4738d16cbdc3128, ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca; no voters:
I20260812 16:58:08.147159 22131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 16:58:08.147893 22131 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [term 1 LEADER]: Becoming Leader. State: Replica: ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca, State: Running, Role: LEADER
I20260812 16:58:08.148545 22131 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.149132 21959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } attempt: 5
Aug 12 16:58:08 dist-test-slave-z8x9 krb5kdc[21655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786553887, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.195.189@KRBTEST.COM for kudu/127.8.195.188@KRBTEST.COM
I20260812 16:58:08.174185 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.149879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.195.188:38231 (local address 127.0.0.1:57182)
0812 16:58:08.150438 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 16:58:08.150464 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 16:58:08.150604 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 16:58:08.152557 (+ 1953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 16:58:08.152564 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 16:58:08.153237 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0812 16:58:08.153502 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.153518 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.155762 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.155770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.157662 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 16:58:08.157684 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.159323 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.159332 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 16:58:08.159635 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.160553 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 16:58:08.160588 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 16:58:08.164344 (+ 3756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 16:58:08.170167 (+ 5823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.170176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.170183 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.170952 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.171529 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 16:58:08.171536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 16:58:08.171543 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 16:58:08.171758 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 16:58:08.172710 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 16:58:08.172719 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 16:58:08.172933 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 16:58:08.173116 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 16:58:08.173581 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372}
I20260812 16:58:08.176031 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:08.150852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.8.195.188:38231)
0812 16:58:08.151011 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.151017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.151034 (+ 17us) server_negotiation.cc:407] Connection header received
0812 16:58:08.151087 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.151093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.151166 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.154383 (+ 3217us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.154555 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.155579 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.157891 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:08.158866 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:08.159019 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:08.164638 (+ 5619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 16:58:08.164648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 16:58:08.164657 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 16:58:08.164712 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 16:58:08.169955 (+ 5243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.171143 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.171152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.171158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.171365 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 16:58:08.171910 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 16:58:08.171918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 16:58:08.171924 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 16:58:08.172426 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 16:58:08.172629 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:08.175118 (+ 2489us) server_negotiation.cc:299] Negotiation successful
0812 16:58:08.175461 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 16:58:08.183871 21959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: Bootstrap starting.
I20260812 16:58:08.191257 21959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: Neither blocks nor log segments found. Creating new log.
I20260812 16:58:08.191156 22176 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.192174 22176 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:08.194854 21959 log.cc:824] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: Log is configured to *not* fsync() on all Append() calls
I20260812 16:58:08.205782 22190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 16:58:08.207971 22190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 16:58:08.208928 21959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: No bootstrap required, opened a new log
W20260812 16:58:08.211931 22030 tablet.cc:2369] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 16:58:08.224427 22071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [term 1 FOLLOWER]: Refusing update from remote peer ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 16:58:08.227108 22200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 16:58:08.227388 22200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 16:58:08.227484 22176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:58:08.228379 21959 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.228894 21959 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdcb508ad10149aa849ea0ca51fdf751, State: Initialized, Role: FOLLOWER
I20260812 16:58:08.229480 21959 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } }
I20260812 16:58:08.238461 21959 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 16:58:08.238489 21915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cdcb508ad10149aa849ea0ca51fdf751"
I20260812 16:58:08.240000 21915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [term 0 FOLLOWER]: Advancing to term 1
I20260812 16:58:08.243561 21915 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca in term 1.
I20260812 16:58:08.244836 22205 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.245441 22205 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.245954 21913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [term 1 FOLLOWER]: Refusing update from remote peer ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 16:58:08.247326 22176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 16:58:08.266656 22205 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.267151 22205 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.270663 22202 mvcc.cc:205] Tried to move back new op lower bound from 7317724726151376896 to 7317724725861986304. Current Snapshot: MvccSnapshot[applied={T|T < 7317724726151376896}]
I20260812 16:58:08.276134 22205 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.276998 22205 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.279431 22170 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.279809 22170 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.239619 21980 master_runner.cc:186] Error getting master registration for 127.8.195.189:39371: OK, instance_id { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" instance_seqno: 1786553887409688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 16:58:08.284569 21980 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 16:58:08.284839 22170 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.285094 22170 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 16:58:08.288702 22131 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.289052 22131 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:08.289486 22131 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca" member_type: VOTER last_known_addr { host: "127.8.195.190" port: 42063 } } peers { permanent_uuid: "cdcb508ad10149aa849ea0ca51fdf751" member_type: VOTER last_known_addr { host: "127.8.195.189" port: 39371 } } peers { permanent_uuid: "a0b9cdd800c546e9b4738d16cbdc3128" member_type: VOTER last_known_addr { host: "127.8.195.188" port: 38231 } } }
I20260812 16:58:08.289784 22131 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca [sys.catalog]: This master's current role is: LEADER
I20260812 16:58:08.292591 22190 catalog_manager.cc:1347] Generated new cluster ID: 3923545c7b7e467f885c5d09b5bc459e
I20260812 16:58:08.292728 22190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 16:58:08.324059 22190 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 16:58:08.326992 22190 catalog_manager.cc:1504] Loading token signing keys...
I20260812 16:58:08.344861 22244 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 16:58:08.349845 22190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce4f5c4e6b6d4d40af3fcb2dc5cfb2ca: Generated new TSK 0
I20260812 16:58:08.351022 22190 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 16:58:08.471571 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.462769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44252 (local address 127.8.195.189:39371)
0812 16:58:08.462958 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.462964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.469276 (+ 6312us) server_negotiation.cc:407] Connection header received
0812 16:58:08.469755 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.469762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.469867 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.469945 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.471316 (+ 1371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44252: BlockingRecv error: recv got EOF from 127.0.0.1:44252 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 16:58:08.472783 22125 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.462353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.8.195.188:38231)
0812 16:58:08.462609 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.462618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.469543 (+ 6925us) server_negotiation.cc:407] Connection header received
0812 16:58:08.469771 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.469776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.469867 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.469945 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.472533 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57196: BlockingRecv error: recv got EOF from 127.0.0.1:57196 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 16:58:08.473014 22127 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.461762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.8.195.190:42063)
0812 16:58:08.462607 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.462613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.471104 (+ 8491us) server_negotiation.cc:407] Connection header received
0812 16:58:08.471368 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.471374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.471453 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.471530 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.472768 (+ 1238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44322: BlockingRecv error: recv got EOF from 127.0.0.1:44322 (error 108)
Metrics: {"server-negotiator.queue_time_us":673}
W20260812 16:58:08.480433 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.478270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.8.195.189:39371)
0812 16:58:08.478497 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.478505 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.478970 (+ 465us) server_negotiation.cc:407] Connection header received
0812 16:58:08.479174 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.479179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.479273 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.479394 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.480160 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44268: BlockingRecv error: recv got EOF from 127.0.0.1:44268 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 16:58:08.486670 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.484669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.8.195.189:39371)
0812 16:58:08.484895 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.484904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.485404 (+ 500us) server_negotiation.cc:407] Connection header received
0812 16:58:08.485617 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.485621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.485685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.485788 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.486473 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44280: BlockingRecv error: recv got EOF from 127.0.0.1:44280 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 16:58:08.496131 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.491885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.8.195.189:39371)
0812 16:58:08.493970 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.493976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.494447 (+ 471us) server_negotiation.cc:407] Connection header received
0812 16:58:08.494744 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.494750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.494863 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.494939 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.495934 (+ 995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44296: BlockingRecv error: recv got EOF from 127.0.0.1:44296 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 16:58:08.506026 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.503843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.8.195.189:39371)
0812 16:58:08.504118 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.504127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.504650 (+ 523us) server_negotiation.cc:407] Connection header received
0812 16:58:08.504871 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.504876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.504966 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.505075 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.505801 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44308: BlockingRecv error: recv got EOF from 127.0.0.1:44308 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
W20260812 16:58:08.519389 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.517208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.8.195.189:39371)
0812 16:58:08.517472 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.517481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.518010 (+ 529us) server_negotiation.cc:407] Connection header received
0812 16:58:08.518231 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.518236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.518325 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.518435 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.519144 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44322: BlockingRecv error: recv got EOF from 127.0.0.1:44322 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 16:58:08.541975 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.539448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.8.195.189:39371)
0812 16:58:08.539637 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.539643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.540230 (+ 587us) server_negotiation.cc:407] Connection header received
0812 16:58:08.540456 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.540465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.540599 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.540717 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.541720 (+ 1003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44336: BlockingRecv error: recv got EOF from 127.0.0.1:44336 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 16:58:08.581347 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.578752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.8.195.189:39371)
0812 16:58:08.578974 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.578980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.579577 (+ 597us) server_negotiation.cc:407] Connection header received
0812 16:58:08.579820 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.579825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.579930 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.580050 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.581018 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44344: BlockingRecv error: recv got EOF from 127.0.0.1:44344 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 16:58:08.661162 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.658640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.8.195.189:39371)
0812 16:58:08.658842 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.658848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.659418 (+ 570us) server_negotiation.cc:407] Connection header received
0812 16:58:08.659612 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.659618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.659722 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.659853 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.660926 (+ 1073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44348: BlockingRecv error: recv got EOF from 127.0.0.1:44348 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 16:58:08.797909 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:08.795784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.8.195.189:39371)
0812 16:58:08.796061 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:08.796069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:08.796369 (+ 300us) server_negotiation.cc:407] Connection header received
0812 16:58:08.796533 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:08.796538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:08.796611 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:08.796700 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:08.797602 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44364: BlockingRecv error: recv got EOF from 127.0.0.1:44364 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 16:58:09.060817 21973 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 16:58:09.059013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.8.195.189:39371)
0812 16:58:09.059207 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.059212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.059554 (+ 342us) server_negotiation.cc:407] Connection header received
0812 16:58:09.059665 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.059670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.059749 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.059857 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 16:58:09.060637 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44380: BlockingRecv error: recv got EOF from 127.0.0.1:44380 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 16:58:09.229645 22200 catalog_manager.cc:1259] Loaded cluster ID: 3923545c7b7e467f885c5d09b5bc459e
I20260812 16:58:09.229764 22200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: loading cluster ID for follower catalog manager: success
I20260812 16:58:09.232640 22200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: acquiring CA information for follower catalog manager: success
I20260812 16:58:09.234478 22200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0b9cdd800c546e9b4738d16cbdc3128: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:58:09.349594 22244 catalog_manager.cc:1259] Loaded cluster ID: 3923545c7b7e467f885c5d09b5bc459e
I20260812 16:58:09.349730 22244 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: loading cluster ID for follower catalog manager: success
I20260812 16:58:09.352376 22244 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: acquiring CA information for follower catalog manager: success
I20260812 16:58:09.354341 22244 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cdcb508ad10149aa849ea0ca51fdf751: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 16:58:09.584604 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:09.577198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.8.195.189:39371)
0812 16:58:09.578117 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.578122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.578140 (+ 18us) server_negotiation.cc:407] Connection header received
0812 16:58:09.578204 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.578208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.578356 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.578468 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:09.579083 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.579976 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.582061 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.582515 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.582614 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:09.583262 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:09.583862 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:09.583979 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:09.584133 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 16:58:09.584198 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":607,"threads_started":1}
I20260812 16:58:09.598882 22376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:09.591437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.8.195.188:38231)
0812 16:58:09.592009 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.592015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.592378 (+ 363us) server_negotiation.cc:407] Connection header received
0812 16:58:09.592595 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.592600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.592753 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.592910 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:09.593457 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.594176 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.596184 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.596583 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.596676 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:09.597406 (+ 730us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:09.598086 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:09.598198 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:09.598377 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 16:58:09.598475 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":270,"threads_started":1}
I20260812 16:58:09.620983 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:09.606011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.8.195.189:39371)
0812 16:58:09.606215 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.606225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.613202 (+ 6977us) server_negotiation.cc:407] Connection header received
0812 16:58:09.613775 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.613784 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.613909 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.613991 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:09.614970 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.616117 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.618482 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.619198 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.619306 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:09.619992 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:09.620146 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:09.620216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:09.620478 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 16:58:09.620571 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 16:58:09.622181 22376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:09.608186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.8.195.188:38231)
0812 16:58:09.611678 (+ 3492us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.611687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.614947 (+ 3260us) server_negotiation.cc:407] Connection header received
0812 16:58:09.615022 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.615028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.615130 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.615216 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:09.616057 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.617159 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.619567 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.620188 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.620349 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:09.621025 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:09.621150 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:09.621275 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:09.621794 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 16:58:09.621883 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3319}
I20260812 16:58:09.622843 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 16:58:09.605376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.8.195.190:42063)
0812 16:58:09.614867 (+ 9491us) server_negotiation.cc:206] Beginning negotiation
0812 16:58:09.614876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 16:58:09.614899 (+ 23us) server_negotiation.cc:407] Connection header received
0812 16:58:09.615023 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 16:58:09.615028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 16:58:09.615366 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 16:58:09.616214 (+ 848us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 16:58:09.616389 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.617490 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.619618 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 16:58:09.620235 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 16:58:09.620746 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 16:58:09.620970 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 16:58:09.621455 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 16:58:09.622062 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0812 16:58:09.622115 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 16:58:09.622210 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9350,"thread_start_us":9230,"threads_started":1}
I20260812 16:58:09.632380 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21671
I20260812 16:58:09.634203 21810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:09.747272 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21671
W20260812 16:58:09.781219 22019 connection.cc:535] server connection from 127.0.0.1:57148 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 16:58:09.781711 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21834
I20260812 16:58:09.782898 21951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:09.889130 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21834
I20260812 16:58:09.916908 8974 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu pid 21980
I20260812 16:58:09.918218 22109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 16:58:10.025720 8974 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKFRmA3/build/asan/bin/kudu with pid 21980
2026-08-12T16:58:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5446 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10940 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (398415 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST